\magnification=\magstep1 \hsize=16truecm \input amstex \TagsOnRight \parindent=20pt \parskip=2pt plus 1.3pt \define\A{{\bold A}} \define\BB{{\bold B}} \define\DD{{\bold D}} \define\T{{\bold T}} \define\U{{\bold U}} \define\({\left(} \define\){\right)} \define\[{\left[} \define\]{\right]} \define\e{\varepsilon} \define\oo{\omega} \define\const{\text{\rm const.}\,} \define\supp {\sup\limits} \define\inff{\inf\limits} \define\summ{\sum\limits} \define\prodd{\prod\limits} \define\limm{\lim\limits} \define\limsupp{\limsup\limits} \define\liminff{\liminf\limits} \define\bigcapp{\bigcap\limits} \define\bigcupp{\bigcup\limits} \define\Cov{\text{\rm Cov}\,} \define\Var{\text{\rm Var}\,} \def\Re{\text{\rm Re}\,} \def\Im{\text{\rm Im}\,} \font\script =cmcsc10 \centerline{\bf ON THE TAIL BEHAVIOUR OF MULTIPLE RANDOM} \centerline{\bf INTEGRALS AND DEGENERATE $U$-STATISTICS} \smallskip \centerline{\it P\'eter Major} \centerline{\it Alfr\'ed R\'enyi Mathematical Institute of the Hungarian Academy of Sciences} \beginsection 1. Introduction In this work the following problem will be investigated: Fix a positive integer $n$, consider $n$ independent, identically distributed random variables $\xi_1,\dots,\xi_n$ on a measurable space $(X,\Cal X)$ with some distribution $\mu$ and define their empirical distribution $\mu_n$ together with its normalization $\sqrt n(\mu_n-\mu)$. Take a function $f(x_1,\dots,x_k)$ of $k$ variables on the $k$-fold product $(X^k,\Cal X^k)$ of the space $(X,\Cal X)$, introduce also the $k$-th power of the normalized empirical measure $\sqrt n(\mu_n-\mu)$ on the space $(X^k,\Cal X^k)$ and define the integral of the function $f$ with respect to this signed product measure. This integral is a random variable, and for all $u>0$ we want to give a good estimate on the probability that it is larger than~$u$. More precisely, we take the integrals not on the whole space, we omit the diagonals $x_s=x_{s'}$, $1\le s,s'\le k$, $s\neq s'$, of the space $X^k$ from the domain of integration. Such a modification of the integral seems to be natural. We shall also be interested in the following generalized version of the above problem. Let us have a nice class of functions $\Cal F$ of $k$ variables on the product space $(X^k,\Cal X^k)$ and consider the integral of all functions of this class with respect to the $k$-fold direct product of our normalized empirical measure. Give a good estimate on the probability that the supremum of these integrals is larger than some number $u>0$. The reader may ask why the above problems deserve a closer study. I found them important, because they may help to solve some important problems in probability theory and mathematical statistics. I met such problems when tried to adapt the method of proof about the Gaussian limit behaviour of the maximum likelihood estimate to some other problems. In the original problem the asymptotic behaviour of the solution of the so-called maximum likelihood equation has to be investigated. The study of this equation is hard in its original form. But by making an appropriate Taylor expansion of the function whose root we are looking for and throwing away its higher order terms we get an approximation whose behaviour can be simply understood. So to describe the limit behaviour of the maximum likelihood estimate it suffices to show that this approximation causes only a negligible error. One would try to apply a similar procedure in more difficult situations. I met some non-parametric maximum likelihood problems, for instance the description of the limit behaviour of the so-called Kaplan--Meyer product limit estimate when such an approach could be applied. But in those problems it was harder to justify that the simplifying approximation causes only a negligible error. To show this the solution of the above mentioned problems were needed. In the non-parametric maximum likelihood estimate problems I met the estimation of multiple (random) integrals played a role similar to the estimation of the coefficients in the Taylor expansion in the study of the maximum likelihood estimate. Although I could apply this approach only in some special cases, I believe that it works in very general situations. But it demands some further work to show this. The problem suggested in this work is interesting and non-trivial even in the special case $k=1$. The solution of the problem in this case leads to some interesting, non-trivial generalization of the fundamental theorem of the mathematical statistics about the difference of the empirical and real distribution of a large sample. The above mentioned problems have a natural counterpart about the behaviour of so-called $U$-statistics, a fairly popular subject in probability theory. The investigation of multiple random integrals and $U$-statistics are closely related, and it turned out that it is useful to consider them simultaneously. Hence both subjects will be discussed in this work. \medskip Let us try to get some feeling what kind of results we can expect. It is useful to observe that for large sample size $n$ the normalized empirical measure $\sqrt n(\mu_n-\mu)$ behaves similarly to a Gaussian random measure. This suggests that in the problems we are interested in similar results should hold as in the case of multiple Gaussian integrals. Hence we may expect that the tail behaviour of the distribution of a $k$-fold random integral with respect to a normalized empirical measure is similar to that of the $k$-th power of a Gaussian random variable with expectation zero and an appropriate variance. Moreover, a similar estimate should hold for the supremum of random integrals of a class of functions under not too restrictive conditions. We may also hope that the methods of the theory of multiple Gaussian integrals can be adapted to the investigation of our problems. The above belief is essentially correct, but there is an essential difference between the behaviour of multiple Gaussian integrals and multiple integrals with respect to a normalized empirical measure. If the variance of a multiple integral with respect to a normalized empirical measure is small, what turns out to be equivalent to the small $L_2$-norm of the function we are integrating, then the behaviour of this integral is different from that of multiple Gaussian integrals with the same variance. In this case the effect of some irregularities of the normalized empirical distribution turns out to be non-negligible, and no good Gaussian approximation holds any longer. Hence some new methods have to be worked out and the hardest problems in our study appear at this point. The precise formulation of the results will be contained in the main part of the work. Beside their proof I also try to explain the main ideas behind them and the notions introduced in their investigation. This work contains some new results, and also the proof of some already rather classical theorems is presented. To make the picture behind the problems more understandable I also discuss their Gaussian counterpart. The proofs apply results from different parts of the probability theory. Papers investigating similar results refer to works dealing with quite different subjects, and this makes their reading rather hard. To overcome this difficulty I tried to work out the details and to present a self-contained discussion even at the price of a longer text. Thus I wrote down (in the main text or in the Appendix) the proof of many interesting and basic results, like results about Vapnik--\v{C}ervonenkis classes, about $U$-statistics and their decomposition to sums of so-called degenerate $U$-statistics, logarithmic Sobolev inequalities, Borell's inequality about homogeneous polynomials of Rademacher functions, etc. I tried to give such an exposition where different parts of the problem are explained as independently of other as possible, and they can be understood in themselves. This work was explained at the probability seminar of the University Debrecen (Hungary). \beginsection 2. Motivation of the investigation. Discussion of some problems Here I try to show by means of some examples why the solution of the problems mentioned in the introduction may be useful in the study of some important probabilistic problems. I try to give a good picture about the main ideas but do not work out all details. Actually, the elaboration of some details omitted would demand hard work. But as the discussion of this section is quite independent of the rest of the paper, these omissions cause no problem in understanding the subsequent part. I start with a short discussion of the maximum likelihood estimate in the simplest case. We study the following problem. Let us have a class of density functions $f(x,\vartheta)$ on the real line depending on a parameter $\vartheta\in R^1$ and observe a sequence of independent random variables $\xi_1(\omega),\dots,\xi_n(\omega)$ with a density function $f(x,\vartheta_0)$, where $\vartheta_0$ is an unknown parameter we want to estimate with the help of the above sequence of random variables. We can carry out this estimation with the help of the maximum likelihood method. It suggests to choose the estimate $\hat \vartheta_n =\hat\vartheta_n(\xi_1,\dots,\xi_n)$ of the parameter $\vartheta_0$ as the number where the density function of the random vector $(\xi_1,\dots,\xi_n)$, i.e.\ the product $$ \prod_{k=1}^n f(\xi_k,\vartheta)=\exp\left\{\sum_{k=1}^n\log f(\xi_k,\vartheta)\right\} $$ takes its maximum. This point can be found as the solution of the so-called maximum likelihood equation $$ \sum_{k=1}^n\frac{\partial}{\partial\vartheta}\log f(\xi_k,\vartheta)=0. \tag2.1 $$ We are interested in the asymptotic behaviour of the random variable $\hat\vartheta_n-\vartheta_0$, where $\hat\vartheta_n$ is the (appropriate) solution of the equation~(2.1). The direct study of this equation is rather hard, but a Taylor expansion of the expression at the left-hand side of~(2.1) around the (unknown) point $\vartheta_0$ helps to give a good and simple approximation of $\hat\vartheta_n$, and it enables us to describe the asymptotic behaviour of $\hat\vartheta_n-\vartheta_0$. This Taylor expansion yields that $$ \allowdisplaybreaks \align &\sum_{k=1}^n\frac{\partial}{\partial\vartheta}\log f(\xi_k,\hat\vartheta_n)= \sum_{k=1}^n\frac{\frac{\partial}{\partial\vartheta} f(\xi_k,\vartheta_0)}{f(\xi_k,\vartheta_0)}\\ &\qquad\qquad+(\hat\vartheta_n-\vartheta_0) \(\sum_{k=1}^n\(\frac{\frac{\partial^2}{\partial\vartheta^2} f(\xi_k,\vartheta_0)}{f(\xi_k,\vartheta_0)}- \frac{\(\frac{\partial}{\partial\vartheta} f(\xi_k,\vartheta_0)\)^2}{f^2(\xi_k,\bar\vartheta_0)} \)\) +O\(n(\hat\vartheta_n-\vartheta_0)^2\) \\ &=\qquad \sum_{k=1}^n\(\eta_k+\zeta_k(\hat\vartheta_n-\vartheta_0)\) +O\(n(\hat\vartheta_n-\vartheta_0)^2\), \tag2.2 \endalign $$ where $$ \eta_k=\frac{\frac{\partial}{\partial\vartheta} f(\xi_k,\vartheta_0)}{f(\xi_k,\vartheta_0)}\quad \text{and}\quad \zeta_k= \frac{\frac{\partial^2}{\partial\vartheta^2} f(\xi_k,\vartheta_0)}{f(\xi_k,\vartheta_0)}- \frac{ \left( \frac{\partial}{\partial\vartheta} f( \xi_k,\vartheta_0)\right)^2}{f^2(\xi_k,\bar\vartheta_0)} $$ for $k=1,\dots,n$. We want to understand the asymptotic behaviour of the (random) expression on the right-hand side of~(2.2). The relation $$ E\eta_k=\int\frac{\frac{\partial}{\partial\vartheta} f(x,\vartheta_0)}{f(x,\vartheta_0)}f(x,\vartheta_0)\,dx =\frac{\partial}{\partial\vartheta}\int f(x,\vartheta_0)\,dx=0 $$ holds, since $\int f(x,\vartheta)\,dx=1$ for all $\vartheta$, and differentiating this relation we get the last identity. Similarly, $E\eta^2_k=-E\zeta_k=\int\frac{\(\frac{\partial}{\partial\vartheta} f(x,\vartheta_0)\right)^2}{f(x,\vartheta_0)}\,dx>0$, \ $k=1,\dots,n$. Hence by the central limit theorem $\chi_n=\frac{1}{\sqrt n}\summ_{k=1}^n\eta_k$ is asymptotically normal with expectation zero and variance $I^2=\int\frac{\left(\frac{\partial}{\partial\vartheta} f(x,\vartheta_0)\right)^2}{f(x,\vartheta_0)}\,dx>0$. In the statistics literature this number $I$ is called the Fisher information. By the laws of large numbers $\frac{1}{n}\summ_{k=1}^n\zeta_k\sim -I^2$. Thus relation (2.2) suggests the approximation $\tilde\vartheta_n= -\frac{\summ_{k=1}^n\eta_k}{\summ_{k=1}^n\zeta_k}$ of the maximum-likelihood estimate $\hat\vartheta_n$, and $\sqrt n(\tilde\vartheta_n-\vartheta_0)$ is asymptotically normal with expectation zero and variance~$I^2$. The random variable $\tilde\vartheta_n$ is not a solution of the equation (2.1), the value of the expression at the left-hand side is of order $O(n(\tilde\vartheta_n-\vartheta_0)^2)=O(1)$ in this point. On the other hand, the derivative of the function at the left-hand side is large in this point, it is greater than $\const n$ with some $\const>0$. This implies that the maximum-likelihood equation has a solution $\hat\vartheta_n$ such that $\hat\vartheta_n-\tilde\vartheta_n=O\(\frac1n\)$. This has the consequence that $\sqrt n(\hat\vartheta_n-\vartheta_0)$ and $\sqrt n(\tilde\vartheta_n-\vartheta_0)$ have the same asymptotic limit behaviour. The previous method can be summarized in the following way: Take a simpler linearized version of the expression we want to estimate by means of an appropriate Taylor expansion, describe the limit distribution of this linearized version and show that the linearization causes only a negligible error. We want to show that such a method also works in more difficult situations. But in some cases it is harder to show that the error we have committed by replacing the original expression by a simpler linearized version is negligible, and to do this we need the solution of the problems mentioned in the introduction. We shall present such an example by studying a fairly popular model of the mathematical statistics, the so-called Kaplan--Meyer method for the estimation of the empirical distribution function with the help of censored data. The following problem is considered. Let $(X_i,Z_i)$, $i=1,\dots,n$, be a sequence of independent, identically distributed random vectors such that the components $X_i$ and $Z_i$ are also independent with distribution functions $F(x)$ and $G(x)$. We want to estimate the distribution function $F$ of the random variables $X_i$, but we cannot observe the variables $X_i$, only the random variables $Y_i=\min(X_i,Z_i)$ and $\delta_i=I(X_i\leq Z_i)$. In other words, we want to solve the following problem. There are certain objects whose lifetime $X_i$ are independent and $F$ distributed. But we cannot observe this lifetime $X_i$, because after a time $Z_i$ the observation must be stopped. We also know whether the real lifetime $X_i$ or the censoring variable $Z_i$ was observed. We make $n$ independent experiments and want to estimate with their help the distribution function~$F$. Kaplan and Meyer, on the basis of some maximum-likelihood estimation type considerations, proposed the following so-called product limit estimator $S_n(u)$ to estimate the unknown survival function $S=1-F$: $$ 1-F_n(u)=S_n(u)=\left\{ \alignedat2 &\prod_{i=1}^n\left(\frac{N(Y_i)}{N(Y_i)+1}\right)^{I(Y_i\leq u, \delta_i=1)} && \text{ if }u\leq\max(Y_1,\dots,Y_n)\\ &0&& \text{ if } u\geq\max(Y_1,\dots,Y_n),\;\delta_n =1,\\ &\text{undefined} &&\text{ if }u\geq\max(Y_1,\dots,Y_n),\;\delta_n =0, \endalignedat\right. \tag2.3 $$ where $$ N(t)=\#\{Y_i,\;\;Y_i>t,\;1\le i \le n\}=\sum_{i=1}^n I(Y_i>t). $$ We want to show that the above estimate (2.3) is really good. For this goal we shall approximate the random variables $S_n(u)$ by some appropriate random variables. To do this first we introduce some notations. Put $$ \aligned H(u) &=P(Y_i\leq u)=1-\bar H(u),\\ \tilde H(u) &=P(Y_i\leq u,\,\delta_i=1),\quad \tilde{\tilde H}(u)=P(Y_i\leq u,\,\delta_i =0) \endaligned \tag2.4 $$ and $$ \aligned H_n(u) &=\frac{1}{n} \sum_{i=1}^n I( Y_i \leq u)\\ \tilde H_n(u) &=\frac1n \sum_{i=1}^n I(Y_i\leq u,\, \delta_i =1), \quad \tilde{\tilde H}_n(u)=\frac{1}{n}\sum_{i=1}^n I( Y_i \leq u, \, \delta_i=0). \endaligned\tag2.5 $$ Clearly $H(u)=\tilde H(u)+\tilde{\tilde H}(u)$ and $ H_n(u)=\tilde H_n(u)+\tilde{\tilde H}_n(u)$. We shall estimate $F_n(u)-F(u)$ for $u\in(-\infty, T]$ if $$ 1-H(T)>\delta \quad \text{with some fixed } \delta>0. \tag2.6 $$ Condition (2.6) implies that there are more than $\frac\delta2n$ sample points $Y_j$ larger than~$T$ with probability almost 1. It has exponentially small probability that this is not the case. This observation helps to show in the subsequent calculations that some events have negligibly small probability. We introduce the so-called cumulative hazard function and its empirical version $$ \Lambda(u)=-\log(1-F(u)), \quad \Lambda_n(u)=-\log(1-F_n(u)). \tag2.7 $$ Since $F_n(u)-F(u)=\exp(-\Lambda(u))\(1-\exp(\Lambda(u)-\Lambda_n(u))\)$ a simple Taylor expansion yields $$ F_n(u)-F(u)=(1-F(u))\left(\Lambda_n(u)-\Lambda(u)\right)+R_1(u),\tag2.8 $$ and it is easy to see that $R_1(u)=O\(\Lambda(u)-\Lambda_n(u))^2\)$. It follows from the subsequent estimations that $\Lambda(u)-\Lambda_n(u)=O(n^{-1/2})$, thus $nR_1(u)=O(1)$. Hence it is enough to investigate the term $\Lambda_n(u)$. We shall show that $\Lambda_n(u)$ has an expansion with $\Lambda(u)$ as the main term plus $n^{-1/2}$ a term which is a linear functional of an appropriate normalized empirical distribution function plus an error term of order $O(n^{-1})$. From~(2.3) it is obvious that $$ \Lambda_n(u)=-\sum_{i=1}^n I(Y_i\leq u, \, \delta_i=1) \log\left(1-\frac{1}{1+N(Y_i)}\right). $$ We can get rid of the unpleasant logarithmic function in this formula by means of the relation $-\log(1-x)=x+O(x^2)$ for small $x$ which yields that $$ \Lambda_n(u)=\sum_{i=1}^n \frac{I(Y_i\leq u, \,\delta_i=1)}{N(Y_i)} +R_2(u)=\tilde{\Lambda}_n(u)+R_2(u), \tag2.9 $$ and the error term $nR_2(u)$ is exponentially small. The expression $\tilde{\Lambda}_n(u)$ is still inappropriate for our purposes. Since the denominators $N(Y_i)=\summ_{j=1}^n I(Y_j>Y_i)$ are dependent for different indices~$i$ we cannot see directly the limit behaviour of $\tilde{\Lambda}_n(u)$. We try to approximate $\tilde{\Lambda}_n(u)$ by a simpler expression. A natural approach would be to approximate the terms $N(Y_i)$ in it by their conditional expectation $(n-1)\bar H(Y_i)=(n-1)(1-H(Y_i))=E(N(Y_i)|Y_i)$. This is a too rough `first order' approximation, but the following `second order approximation' will be sufficient for our goals. Put $$ N(Y_i)=\sum_{j=1}^n I(Y_j>Y_i)=n\bar H(Y_i) \(1+ \frac{\summ_{j=1}^nI(Y_j>Y_i)-n\bar H(Y_i)}{n\bar H(Y_i)}\) $$ and express the terms $\frac1{N(Y_i)}$ in the sum defining $\tilde \Lambda_n$ by means of the relation $\frac1{1+z}=\summ_{k=0}^\infty (-1)^kz^k=1-z+\e(z)$ with the choice $z=\frac{\summ_{j=1}^nI(Y_j>Y_i)-n\bar H(Y_i)}{n\bar H(Y_i)}$. As $|\e(z)|<2z^2$ for $|z|<\frac{1}{2}$ we get that $$ \align \tilde{\Lambda}_n(u)& =\sum_{i=1}^n\frac{I(Y_i\leq u,\,\delta_i=1)} {n\bar H(Y_i)}\(1+\sum_{k=1}^\infty\(- \frac{\summ_{j=1}^n I(Y_j>Y_i)-n\bar H(Y_i)} {n\bar H(Y_i)}\)^k\)\\ &=\sum_{i=1}^n\frac{I(Y_i\leq u,\,\delta_i=1)} {n\bar H(Y_i)}\(1-\frac{\summ_{j=1}^n I(Y_j>Y_i)-n\bar H(Y_i)} {n\bar H(Y_i)}\)+R_3(u) \tag2.10 \\ &=2A(u)-B(u)+R_3(u), \endalign $$ where $$ A(u)=A(n,u)=\sum_{i=1}^n\frac{I(Y_i\leq u,\,\delta_i=1)}{n\bar H(Y_i)} $$ and $$ B(u)=B(n,u)=\sum_{i=1}^n \sum_{j=1}^n\frac {I(Y_i\leq u,\,\delta_i=1)I(Y_j>Y_i)}{n^2\bar H^2(Y_i)}. $$ It can be proved by means of standard methods that $nR_3(u)$ is exponentially small. Thus from~(2.9) and~(2.10) we get that $$ \Lambda_n(u)=2A(u)-B(u)+\text{negligible error.}\tag2.11 $$ This means that to solve our problem we have to describe the asymptotic behaviour of the random variables $A(u)$ and $B(u)$. We can get a better insight into their behaviour by rewriting the sum $A(u)$ as an integral with respect to an empirical measure and the double sum $B(u)$ as a two-fold integral with respect empirical measures. These integrals can be rewritten as sums of random integrals with respect to normalized empirical measures and deterministic measures. In such a way we get a representation of $\Lambda_n(u)$ in the form of a sum whose terms can be well understood. Let us write $$ \align A(u)&=\int_{-\infty}^{+\infty}\frac{I(y\leq u)}{1-H(y)}\,d\tilde H_n(y),\\ B(u) &=\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty} \frac{I(y\leq u)I(x>y)}{\(1-H(y)\)^2}\,dH_n(x) d\tilde H_n(y). \endalign $$ To rewrite the term $B(u)$ in a form better for our purposes observe that $$ \align H_n(x)\tilde H_n(y)&=H(x)\tilde H(y)+H(x)(\tilde H_n(y)-\tilde H(y)) +(H_n(x)-H(x))\tilde H(y)\\ &\qquad+(H_n(x)-H(x))(\tilde H_n(y)-\tilde H(y)). \endalign $$ Hence it can be written in the form $B(u)=B_1(u)+B_2(u)+B_3(u)+B_4(u)$, where $$ \align B_1(u)&=\int_{-\infty}^u\int_{-\infty}^{+\infty} \frac{I(x>y)}{\(1-H(y)\)^2}\,dH(x)\,d\tilde H(y)\;,\\ B_2(u) &=\frac{1}{\sqrt n}\int_{-\infty}^u\int_{-\infty}^{+\infty} \frac{I(x>y)}{\(1-H(y)\)^2}\,dH(x)\,d\(\sqrt n (\tilde H_n(y)-\tilde H(y))\),\\ B_3(u)&=\frac1{\sqrt n}\int_{-\infty}^u \int_{-\infty}^{+\infty}\frac{I(x>y)}{\(1-H(y)\)^2} \,d\(\sqrt n\(H_n(x)-H(x)\)\)\,d\tilde H(y)\;,\\ B_4(u)&=\frac 1n \int_{-\infty}^u\int_{-\infty}^{+\infty} \frac{I(x>y)}{\(1-H(y)\)^2}\,d\(\sqrt n\(H_n(x)-H(x)\)\)\, d\(\sqrt n(\tilde H_n(y)-\tilde H(y))\). \endalign $$ In the above decomposition of $B(u)$ the term $B_1$ is a deterministic function, $B_2$, $B_3$ are linear functionals of empirical processes and $B_4$ is a nonlinear functional of empirical processes. The deterministic term $B_1(u)$ can be calculated explicitly. Indeed, $$ B_1(u)=\int_{-\infty}^u\int_{-\infty}^{+\infty} \frac{I(x>y)}{\(1-H(y)\)^2}\,dH(x) d\tilde H(y)= \int_{-\infty}^u\frac{d\tilde H(y)}{1-H(y)}. $$ Then the relations $\tilde H(u)=\int_{-\infty}^u\(1-G(t)\)\,dF(t)$ and $1-H = (1-F)(1-G)$ imply that $$ B_1(u)=\int_{-\infty}^u\frac{dF(y)}{1-F(y)}= -\log(1-F(u))=\Lambda(u).\tag2.12 $$ Observe that $$ \aligned A(u) &=\int_{-\infty}^u\frac{d\,\tilde H_n(y)}{1-H(y)}\\ &=\int_{-\infty}^u\frac{d\tilde H(y)}{1-H(y)}+ \frac1{\sqrt n}\int_{-\infty}^u \frac{d \(\sqrt n (\tilde H_n(y)-\tilde H(y))\)}{1-H(y)}\\ &=B_1(u)+B_2(u). \endaligned\tag2.13 $$ From relation~(2.11) using~(2.12) and~(2.13) it follows that $$ \Lambda_n(u)-\Lambda(u)=B_2(u)-B_3(u)-B_4(u)+\text{negligible error.} \tag2.14 $$ Integrating $B_2$ and $B_3$ in the variable $x$ and then integrating by parts $B_2$ we get that $$ \align B_2(u)&=\frac1{\sqrt n}\int_{-\infty}^{u} \frac{d\(\sqrt n(\tilde H_n(y)-\tilde H(y))\)}{1-H(y)}\\ &=\frac{\sqrt n\(\tilde H_n(u)-\tilde H(u)\)}{\sqrt n(1-H(u))}- \frac1{\sqrt n}\int_{-\infty}^{u} \frac{\sqrt n(\tilde H_n(y)-\tilde H(y))}{\(1-H(y)\)^2}\,dH(y)\\ B_3(u)&=\frac1{\sqrt n}\int_{-\infty}^u \frac{\sqrt n\(H_n(y)-H(y)\)}{\(1-H(y)\)^2}\,d\tilde H(y). \endalign $$ Using the above forms of $B_2$ and $B_3$,~(2.12) we can write $$ \aligned \sqrt n\(\Lambda_n(u)-\Lambda(u)\) &=\frac{\sqrt n\(\tilde H_n(u)-\tilde H(u)\)}{1-H(u)}-\int_{-\infty}^u \frac{\sqrt n(\tilde H_n(y)-\tilde H(y))}{\(1-H(y)\)^2}\,dH(y)\\ &\qquad+\int_{-\infty}^u\frac{\sqrt n\(H_n(y)-H(y)\)}{\(1-H(y)\)^2} \,d\tilde H(y)\\ &\qquad-\sqrt n B_4(u)+\text{negligible error.} \endaligned\tag2.15 $$ Formula (2.15) almost agrees with the statement we wanted to prove. Here we expressed the normalized error $\sqrt n\(\Lambda_n(u)-\Lambda(u)\)$ as a sum of linear functionals of normalized empirical measures plus some negligible error terms and the error term $\sqrt nB_4(u)$. So to get a complete proof it is enough to show that $\sqrt nB_4(u)$ also yields a negligible error. But $B_4(u)$ is a double integral of a bounded function (here we apply again formula (2.6)) with respect to a normalized empirical measure. Hence to bound this term we need a good estimate of multiple stochastic integrals (with multiplicity~2) and this is just the problem formulated in the introduction. The estimate we need here follows from Theorem~8.1 of the present work. Let us remark that the problem discussed here corresponds to the estimation of the coefficient of the second term in the Taylor expansion considered in the study of the maximum likelihood estimation. One may worry a little bit how to bound $B_4(u)$ with the help of estimations of double stochastic integrals, since in the definition of $B_4(u)$ we integrate by different normalized empirical processes in the two coordinates. But this is a not too difficult technical problem, it can be simply overcome for instance by rewriting the integral as a double integral with respect to the empirical process $\(\sqrt n\(H_n(x)-H(x)\), \sqrt n\(\tilde H_n(y)-\tilde H(y)\)\)$ in the space $R^2$. By working out the details of the above calculation we get that the linear functional $B_2(u)-B_3(u)$ of normalized empirical processes yields a good estimate on the expression $\sqrt n(\Lambda_n(u)-\Lambda(u))$ for a fixed parameter~$u$. But we want to prove somewhat more, we want to get an estimate uniform in the parameter~$u$, i.e. to show that even the random variable $\supp_{u\le T}\left|\sqrt n(\Lambda_n(u)-\Lambda(u))-B_2(u)+B_3(u)\right|$ is small. This can be done by making estimates uniform in the parameter~$u$ in all steps of the above calculation. There appears only one difficulty when trying to carry out this program. Namely, we need an estimate on $\supp_u |B_4(u)|$, i.e. we have to bound the supremum of multiple random integrals with respect to a normalized random measure for a nice class of kernel functions. This can be done, but at this point the second problem mentioned in the introduction appears. This difficulty can be overcome by means of Theorem~8.2 of this work. Thus we can find the limit behaviour of the Kaplan--Meyer estimate by means of an appropriate expansion. The steps of this investigation are fairly standard, the only hard part is the solution of the problems mentioned in the introduction. We expect that such a method also works in much more general situation. This may justify a detailed study of the problems considered in this work. I finish this section with a remark of Richard Gill he made in a personal conversation after my talk on this subject at a conference. He told that this approach had given a complete proof about the limit behaviour of this estimate, but it had exploited the explicit formula given in the Kaplan--Meyer estimate. He missed the application of an argument based on the non-parametric maximum likelihood character of this estimate. This was a completely justified remark, since if we do not restrict our attention to this problem, but try to generalize it to general non-parametric maximum likelihood estimates, then we have to understand how the maximum likelihood character can be exploited. I believe that this can be done, but it demands further studies. \beginsection 3. Some estimates about sums of independent random variables We need some results about the distribution of sums of independent random variables bounded by a constant with probability one. Later only the results about sums of independent and identically distributed variables will be interesting for us, but since these results can be generalized without any effort to sums of not necessarily identically distributed random variables here we shall drop the condition about the identical distribution of the summands. We are interested in the question when these estimates give such a good bound as the central limit theorem suggests, and what can be told if this is not the case. More explicitly, we consider the following problem: Let $X_1,\dots,X_n$ be independent random variables $EX_j=0$, $\Var X_j=\sigma_j^2$, $1\le j\le n$, and take the random sum $S_n=\summ_{j=1}^nX_j$ and its variance $\Var S_n=V_n^2= \summ_{j=1}^n\sigma_j^2$. We want to get a good bound on the probability $P(S_n>x V_n)$. The central limit theorem would suggest that under general conditions an upper bound of the order $1-\Phi(x)$ should hold for this probability where $\Phi(x)$ denotes the standard normal distribution function. Since the standard normal distribution function satisfies the inequality $\(\frac1x-\frac1{x^3}\)\frac{e^{-x^2/2}}{\sqrt{2\pi}} <1-\Phi(x)<\frac1x\frac{e^{-x^2/2}}{\sqrt{2\pi}}$ for all $x>0$ it is natural to ask when the probability $P(S_n >xV_n)$ is comparable with the value $e^{-x^2/2}$. More generally, we say that we have a Gaussian type estimate for the probability $P(S_n >xV_n)$ if it can be bounded by $e^{-Cx^2}$ with some constant $C$ separated from zero. First we discuss Bernstein's inequality which tells for which values $x$ the probability $P(S_n>xV_n)$ satisfies a Gaussian type estimate. Such an estimate holds (for sums of random variables bounded by 1) if $x\le \const V_n$. For $x\ge \const V_n$ Bernstein's inequality yields almost no improvement if we have a better bound on the variance $V_n$ of the sum $S_n$. Another estimate, Bennett's inequality yields a slight improvement, and as an example presented before this result shows it cannot be essentially improved without imposing some additional conditions. The main difficulties we meet in this paper are closely related to the weakness of the estimates we have for the probability of the event that a sum of independent random variables is larger than some value when this probability does not satisfy a Gaussian type estimate because of the small variance of the sum. Let us formulate Bernstein's inequality. In its usual formulation a real number~$M$ is introduced and it is assumed that the terms in the sum we investigate are bounded by this number. But since the problem can be simply reduced to the special case $M=1$ we shall only deal with this special case. \medskip\noindent {\bf Theorem 3.1 (Bernstein's inequality).} {\it Let $X_1,\dots,X_n$ be independent random variables, $P(|X_j|\le1)=1$, $EX_j=0$, $1\le j\le n$. Put $\sigma_j^2=EX_j^2$, $1\le j\le n$, $S_n=\summ_{j=1}^n X_j$ and $V_n^2=\Var S_n=\summ_{j=1}^n\sigma_j^2$. Then $$ P\(S_n>xV_n\)\le\exp\left\{-\frac{x^2}{2\(1+\frac13\frac x{V_n}\)} \right\} \quad\text{for all }x>0. \tag3.1 $$ }\medskip\noindent {\it Proof of Theorem 3.1.} Let us give a good bound on the exponential moments $Ee^{tS_n}$ for some appropriate parameters $t>0$. We can write $Ee^{tX_j}=\summ_{k=0}^\infty \frac{t^k}{k!}EX_j^k\le 1+\frac{t^2\sigma_j^2}2\(1+\summ_{k=1}^\infty \frac {2t^{k}}{(k+2)!}\) \le 1+\frac{t^2\sigma_j^2}2 \(1+\summ_{k=1}^\infty 3^{-k}t^{k}\) = 1+\frac{t^2\sigma_j^2}2\frac1{1-\frac t3} \le\exp\left\{\frac{t^2\sigma_j^2}2\frac1{1-\frac t3}\right\}$ if $0\le t<3$. Hence $Ee^{tS_n}=\prodd_{j=1}^n Ee^{tX_j}\le \exp\left\{\frac{t^2V_n^2}2\frac1{1-\frac t3}\right\}$ for $0\le t<3$. The above relation implies that $$ P\(S_n>xV_n\)=P(e^{tS_n}>e^{txV_n})\le Ee^{tS_n}e^{-txV_n}\le \exp\left\{\frac{t^2V_n^2}2\frac1{1-\frac t3}-txV_n\right\} $$ if $0\le t<3$. Choose the number $t$ in this inequality as the solution of the equation $t^2V_n^2\frac1{1-\frac t3}=txV_n$, i.e. put $t=\frac x{V_n+\frac x3}$. Then $0\le t<3$, and we get that $P(S_n>xV_n)\le e^{-txV_n/2}= \exp\left\{-\frac{x^2}{2\(1+\frac13\frac x{V_n}\)}\right\}$. \medskip If the random variables $X_1,\dots,X_n$ satisfy the conditions of the Bernstein inequality then also the random variables $-X_1,\dots,-X_n$ satisfy them. By applying the above result in both cases we get that $P(|S_n|>xV_n)\le2\exp\left\{-\frac{x^2}{2\(1+\frac13\frac x{V_n}\)} \right\}$ under the conditions of the Bernstein inequality. Bernstein's inequality states that for all $\e>0$ there is some sufficiently small number $\alpha(\e)>0$ such that in the case $\frac x{V_n}<\alpha(\e)$ $P(S_n>xV_n)\le e^{-(1-\e)x^2/2}$. Beside this, for all fixed numbers $A>0$ there is some constant $C=C(A)>0$ such that in the case $\frac x{V_n}xV_n)\le e^{-Cx^2}$ holds. This can be interpreted as a Gaussian type estimate for the probability $P(S_n>xV_n)$. On the other hand, if $\frac x{V_n}$ is very large, then the Bernstein inequality yields a much worse estimate. The next example shows that this is not because of its weakness. There are sequences of independent, identically distributed random variables $X_1,\dots,X_n$ bounded by one and with expectation zero such that with the notations $S_n=\summ_{j=1}^nX_j$, $\sigma^2=EX_j^2$, $V_n^2=\summ_{j=1}EX_j^2=n\sigma^2$ the probability $P(S_n>xV_n)$ is relatively large if $\frac x{V_n}$ is large, it is much larger than the value suggested by the normal approximation. This example will be interesting for us mainly for the sake of some orientation. Hence I do not try to formulate it in such a general form as it could be done or to give the best possible constants in it. The method of proof shows that a wide class of examples could be constructed with similar properties. In the following discussion it will be convenient to replace the number $x$ by $y=xV_n=\sqrt n\sigma x$. \medskip\noindent {\bf Example 3.2.} {\it Let us fix some positive integer $n$, real numbers $y\ge200$ and $1>\sigma^2>0$ such that $n>16y>64n\sigma^2$. Put $V_n^2=n\sigma^2$ and take a sequence of independent, identically distributed random variables $X_1,\dots,X_n$ such that $P(X_j=1)=P(X_j=-1)=\frac{\sigma^2}2$, and $P(X_j=0)=1-\sigma^2$. Put $S_n=\summ_{j=1}^n X_j$. Then $ES_n=0$, $\Var S_n=V_n^2$, and $$ P(S_n>y)>A\exp\left\{-By\log \frac y {V^2_n}\right\} $$ with some universal constants $A>0$ and $B>0$. We can choose for instance $A=\frac12$, $B=\frac{22}5$ in this inequality.} \medskip Here I shall give a proof of the statement of Example 3.2. Let me remark that in the work~[23] I gave a simpler and more elementary proof of this result under the name Example~2.4. \medskip\noindent {\it Proof of the statement of Example 3.2.} In the proof some ideas of the large deviation theory will be applied. Let us introduce the measure $\mu$, $\mu(\{1\})=\mu(\{-1\})=\frac{\sigma^2}2$, $\mu(\{0\})=1-\sigma^2$ on the real line, which is actually the distribution of the random variables $X_j$, together with its conjugates $\mu_t$, $\mu_t(\,dx)=\frac{e^{tx}}{\frac{\sigma^2} 2(e^t+e^{-t})+1-\sigma^2}\mu(\,dx)$, $x\in R^1$, for all real numbers~$t$. Let $\mu^{(n)}$ denote the $n$-fold convolution of the measure $\mu$ and $\mu^{(n)}_t$ the $n$-fold convolution of the measure $\mu_t$ with itself. Then $P(S_n>y)=\mu^{(n)}((y,\infty))$, and it is not difficult to see (and it is a well-known fact in the theory of large deviations) that $\mu^{(n)}(A)=\(\frac {\sigma^2}2(e^t+e^{-t})+1-\sigma^2\)^n\int_A e^{-tu}\mu^{(n)}_t(\,du)$ for all measurable sets $A\subset R^1$. Let us consider the above defined measures $\mu_t$ and $\mu_t^{(n)}$ with $t=\log\frac {4y}{n\sigma^2}$. I claim that $\mu^{(n)}_t([y,\frac{11}5y])\ge\frac12$. To show this let us consider $n$ independent $\mu_t$ distributed, independent random variables $\xi_1,\dots,\xi_n$, and estimate their expected value and variance. We have $E\xi_j=\frac{\frac{\sigma^2}2(e^t-e^{-t})} {\frac{\sigma^2}2(e^t+e^{-t})+1-\sigma^2}$ for all $1\le j\le n$, and since $1\le\frac{\sigma^2}2(e^t+e^{-t})+1-\sigma^2\le1+\sigma^2e^t =1+4\frac yn\le\frac54$, and beside this we get with the help of the estimate $e^{-t}=e^te^{-2t}=e^t\(\frac{n\sigma^2}{4y}\)^2\le \frac14e^t$ the inequality $\frac32\frac yn=\frac38\sigma^2e^t \le\frac{\sigma^2}2 (e^t-e^{-t})\le\frac{\sigma^2}2e^t=2\frac yn$, hence $\frac65\frac yn\le E\xi_j\le2\frac yn$. Similarly, $\Var\xi_j \le E\xi_j^2=\frac{\frac{\sigma^2}2(e^t+e^{-t})}{\frac{\sigma^2} 2(e^t+e^{-t})+1-\sigma^2}\le 4\frac yn$. The above estimates together with the Chebishev inequality imply that $\mu^{(n)}_t([y, \frac{11}5y])=P\(y\le\summ_{j=1}^n\xi_j\le\frac{11}5y\) \ge1-P\(\left|\summ_{j=1}^n(\xi_j-E\xi_j)\right|>\frac y5\)\ge 1-\frac{100y}{y^2}\ge\frac12$. This inequality together with the relation between the measures $\mu^{(n)}$ and $\mu_t^{(n)}$ imply that $$ \allowdisplaybreaks \align P(S_n>y)&=\mu^{(n)}([y,\infty]) \ge\mu^{(n)}\(\[y,\frac{11}5y\]\)\\ &=\(\frac{\sigma^2}2(e^t+e^{-t})+1-\sigma^2\)^n \int\limits_y^{11y/5} e^{-tu}\mu^{(n)}_t(\,du) \ge e^{-11ty/5}\mu_t^{(n)}\(\[y,\frac{11}5y\]\) \\ &\ge\frac12e^{-11ty/5}=\frac12\exp\left\{-\frac{11}5y \log\frac {4y}{V_n^2}\right\} \ge\frac12\exp\left\{-\frac{22}5y \log\frac y{V_n^2}\right\}. \endalign $$ \medskip In the case $y>V_n^2$ the Bernstein inequality yields the estimate $P(S_n>y)\le e^{-\alpha y}$ with some universal constant $\alpha>0$, while the above example shows that we can expect at most an additional logarithmic factor in the exponent of the upper bound in an improvement of this estimate. The following result, called Bennett's inequality shows that such an improvement is really possible. \medskip\noindent {\bf Theorem 3.3 (Bennett's inequality).} {\it Let $X_1,\dots,X_n$ be independent random variables, $P(|X_j|\le1)=1$, $EX_j=0$, $1\le j\le n$. Put $\sigma_j^2=EX_j^2$, $1\le j\le n$, $S_n=\summ_{j=1}^n X_j$ and $V_n^2=\Var S_n=\summ_{j=1}^n\sigma_j^2$. Then $$ P(S_n>y)\le\exp\left\{-V^2_n\[\(1+\frac y{V^2_n}\) \log\(1+\frac y{V^2_n}\)-\frac y{V^2_n}\]\right\}\quad\text{for all }y>0. \tag3.2 $$ As a consequence, for all $\e>0$ there exists some $B=B(\e)>0$ such that $$ P\(S_n>y\)\le\exp\left\{-(1-\e)y\log \frac y{V^2_n} \right\}\quad\text{if } y>BV_n^2, \tag3.3 $$ and there exists some positive constant $K>0$ such that $$ P\(S_n>y\)\le\exp\left\{-Ky\log \frac y{V^2_n} \right\}\quad\text{if }y>2V_n^2. \tag3.4 $$ }\medskip\noindent {\it Proof of Theorem 3.3.}\/ We have $$ Ee^{tX_j}=\summ_{k=0}^\infty\frac{t^k}{k!}EX_j^k\le 1+\sigma_j^2\summ_{k=2}^\infty\frac {t^k}{k!}=1+\sigma_j^2 \(e^t-1-t\)\le e^{\sigma_j^2(e^t-1-t)}, \quad 1\le j\le n, $$ and $Ee^{tS_n}\le e^{V_n^2(e^t-1-t)}$ for all $t\ge0$. Hence $P(S_n>y)\le e^{-ty}Ee^{tS_n}\le e^{-ty+V_n^2(e^t-1-t)}$ for all $t\ge0$. We get relation (3.2) from this inequality with the choice $t=\log\(1+\frac y{V^2_n}\)$. (This is the place of minimum of the function $-ty+V_n^2(e^t-1-t)$ for fixed $y$ in the parameter~$t$.) Relation (3.2) and the observation $\limm_{u\to\infty}\frac{(u+1)\log(u+1)-u}{u\log u}=1$ with the choice $u=\frac y{V_n^2}$ imply formula~ (3.3). Because of relation (3.3) to prove formula (3.4) it is enough to check it for $2\le \frac y{V_n^2}\le B$ with some sufficiently large constant $B>0$. In this case relation (3.4) follows directly from formula (3.2). This can be seen for instance by observing that the expression $\frac {V^2_n \[\(1+\frac y{V^2_n}\) \log\(1+\frac y{V^2_n}\)-\frac y{V^2_n}\]}{y\log\frac y{V^2_n}}$ is a continuous and positive function of the variable $\frac y{V_n^2}$ in the interval $2\le \frac y{V_n^2}\le B$, hence its minimum in this interval is strictly positive. \medskip Let us make a short comparison between Bernstein's and Bennett's inequality. Both results deal with the estimation of the probability $P(S_n>y)$, and their proofs are also very similar. In both cases first an estimate is given for the moment generating functions $R_j(t)=Ee^{tX_j}$ of the summands~$X_j$. In Bennett's inequality a better estimate is given for them. (The worst case we have to handle is when $P(X_j=1)=\e_j$, $P\(X_j=-\frac{e_j}{1-\e_j}\)=1-\e_j$, and $\e_j+\frac{\e_j^2}{1-\e_j}=\sigma_j^2$. In this case the proof of Bennett's inequality contains an almost optimal estimate, while the estimate in Bernstein's inequality is weaker. In this estimate we are satisfied to give a good estimate for the first three coefficients in the Taylor expansion of the function $R_j(t)$.) With the help of this estimate a bound is given on the probability we are interested in which depends on the parameter~$t$. In the proof of Bennett's inequality this parameter~$t$ is chosen optimally, while in Bernstein's inequality only an asymptotically optimal choice is taken. As a consequence, Bennett's inequality yields a sharper estimate. Actually Bernstein's inequality can be deduced from it. On the other hand, Bernstein's inequality gives a good, `visible' bound for the probability $P(S_n>y)$ for not too large values of the number~$y$ which suffices for our purposes, while the magnitude of the estimate given by Bennett's inequality for small~$y$ cannot be directly seen. For large $y$ Bennett's yields a better estimate, but this improvement seems to have a smaller importance. I finish this section with another estimate due to Hoeffding which later will be useful for us when we want to carry out certain symmetrization arguments. \medskip\noindent {\bf Theorem 3.4 (Hoeffding's inequality).} {\it Let $\e_1,\dots,\e_n$ be independent random variables, $P(\e_j=1)=P(\e_j=-1)=\frac12$, $1\le j\le n$, and let $a_1,\dots,a_n$ be arbitrary real numbers. Put $V=\summ_{j=1}^na_j\e_j$. Then $$ P(V>y)\le\exp\left\{-\frac{y^2}{2\sum_{j=1}^na_j^2 }\right\}\quad \text{for all }y>0. \tag3.5 $$ }\medskip\noindent {\it Remark:}\/ Clearly $EV=0$ and $\Var V=\summ_{j=1}^n a_j^2$, hence Hoeffding's inequality yields such an estimate for $P(V>y)$ which the central limit theorem suggests. This estimate holds for all real numbers $a_1,\dots,a_n$. \medskip\noindent {\it Remark 2:}\/ If we consider the Rademacher functions $r_k(x)$, $r_k(x)=1$ if $(2j-1)2^{-k}\le x<2j2^{-k}$ and $r_k(x)=-1$ if $2(j-1)2^{-k}\le x<(2j-1)2^{-k}$, $1\le j\le 2^k$, for all $k=1,2,\dots$, as random variables on the probability space $\Omega=[0,1]$ with the Borel $\sigma$-algebra and the Lebesgue measure as probability measure on the interval $[0,1]$, then they are independent random variables with the same distribution as the random variables $\e_1,\dots,\e_n$ considered in Theorem~3.4. Therefore such results which deal with random variables of this type are also called results about Rademacher functions in the literature. At some points we shall also use this terminology. \medskip\noindent {\it Proof of Theorem 3.4.} Let us give a good bound on the exponential moment $Ee^{tV}$ for all $t>0$. We have $Ee^{tV}=\prodd_{j=1}^nEe^{ta_j\e_j}= \prodd_{j=1}^n\frac{\(e^{a_jt}+e^{-a_jt}\)}2$, and $\frac{\(e^{a_jt}+e^{-a_jt}\)}2=\summ_{k=0}^\infty \frac{a_{j}^{2k}} {(2k)!}t^{2k}\le \summ_{k=0}^\infty \frac {(a_jt)^{2k}}{2^{k}k!}=e^{a_j^2t^2/2}$, since $(2k)!\ge 2^k k!$ for all $k\ge0$. This implies that $Ee^{tV}\le \exp\left\{\frac{t^2}2\summ_{j=1}^n a_j^2\right\}$. Hence $P(V>y)\le\exp\left\{-ty+\frac{t^2}2\summ_{j=1}^n a_j^2\right\}$, and we get relation (3.5) with the choice $t=y\(\summ_{j=1}^n a_j^2\)^{-1}$. \medskip \beginsection 4. On the supremum of a nice class of partial sums This section contains a result about the behaviour of the supremum of random integrals with respect to a normalized empirical measure in the special case when only one-fold integrals are considered. First we present an equivalent version of it about the supremum of a nice class of sums of independent, identically distributed random variables. We also discuss some natural problems related to them. In particular, we are interested in the question how restrictive the conditions of these results are. Also the natural Gaussian counterpart of these results will be given, but the proofs are postponed to a later section. To formulate our results first we introduce the following notion. \medskip\noindent {\bf Definition of $L_p$-dense classes of functions.} {\it Let us have a measurable space $(Y,\Cal Y)$ and a set $\Cal G$ of $\Cal Y$ measurable real valued functions on this space. We call $\Cal G$ an $L_p$-dense class of functions, $1\le p<\infty$, with parameter $D$ and exponent $L$ if for all numbers $1\ge\e>0$ and probability measures $\nu$ on the space $(Y,\Cal Y)$ there exists a finite $\e$-dense subset $\Cal G_{\e,\nu}=\{g_1,\dots,g_m\} \subset \Cal G$ in the space $L_p(Y,\Cal Y,\nu)$ consisting of $m\le D\e^{-L}$ elements, i.e. there exists such a set $\Cal G_{\e,\nu}\subset \Cal G$ for which $\inff_{g_j\in \Cal G_{\e,\nu}}\int |g-g_j|^p\,d\nu<\e^p$ for all functions $g\in \Cal G$. (Here the set $\Cal G_{\e,\nu}$ may depend on the measure $\nu$, but its cardinality is bounded by a number depending only on $\e$.)} \medskip Now we formulate the following \medskip\noindent {\bf Theorem 4.1.} {\it Let us have a sequence of iid. random variables $\xi_1,\dots,\xi_n$, $n\ge2$, taking values on a measurable space $(X,\Cal X)$ with some distribution $\mu$ together with an $L_2$-dense class $\Cal F$ of functions of countable cardinality with some parameter $D$ and exponent $L\ge1$ on the space $(X,\Cal X)$ which satisfies the conditions $$ \align \|f\|_\infty&=\supp_{x\in X}|f(x)|\le 1, \qquad \text{for all } f\in \Cal F \tag4.1 \\ \|f\|_2^2&=\int f^2(x) \mu(\,dx)\le \sigma^2 \qquad \text{for all } f\in \Cal F \tag4.2 \endalign $$ with some constant $\sigma>0$, and $$ \int f(x)\mu(\,dx)=0 \quad \text{for all } f\in\Cal F \tag4.3 $$ Define the normalized partial sums $S_n(f)=\frac1{\sqrt n} \summ_{k=1}^n f(\xi_k)$ for all $f\in \Cal F$ and introduce the number $\beta=\max\(\frac{\log D}{\log n},0\)$, where $D$ is the parameter of the $L_2$-dense class $\Cal F$. There exist some constants $C>0$, $\alpha>0$ and $M>0$ such that the supremum of the normalized random sums $S_n(f)$, $f\in \Cal F$, satisfies the inequality $$ \aligned P\(\supp_{f\in\Cal F}|S_n(f)|\ge u\)&\le CD \exp\left\{-\alpha \(\frac u{\sigma}\)^2\right\} \\ &\qquad \text{if}\quad \sqrt n\sigma^2\ge u\ge \sqrt M(L+\beta)^{3/4}\sigma\log^{1/2}\frac2\sigma, \endaligned \tag4.4 $$ with the number $\beta$ defined in this theorem, and the numbers $D$ and $L$ in formula~(4.4) agree with the parameter and exponent of the $L_2$-dense class~$\Cal F$.} \medskip The condition about the countable cardinality of $\Cal F$ can be weakened. For this goal we introduce the notion of countable approximability. For the sake of later applications it will be formulated more generally than needed in the present context. \medskip\noindent {\bf Definition of countably approximable classes of random variables.} {\it Let a class of random variables $U(f)$, $f\in \Cal F$, indexed by a class of functions on a measure space $(Y,\Cal Y)$ be given. We say that this class of random variables $U(f)$, $f\in\Cal F$, is countably approximable if there is a countable subset $\Cal F'\subset \Cal F$ such that for all numbers $u>0$ the sets $A(u)=\{\oo\:\supp_{f\in \Cal F}|U(f)(\oo)|\ge u\}$ and $B(u)=\{\oo\:\supp_{f\in \Cal F'} |U(f)(\oo)|\ge u\}$ satisfy the identity $P(A(u)\setminus B(u))=0$.} \medskip Clearly, $B(u)\subset A(u)$. In the above definition we demanded that for all $u>0$ the set $B(u)$ should be almost as large as $A(u)$. The following corollary of Theorem~4.1 holds. \medskip\noindent {\bf Corollary of Theorem~4.1.} {\it Let a class of functions $\Cal F$ satisfy the conditions of Theorem~4.1 with the only exception that instead of the condition about the countable cardinality of $\Cal F$ it is assumed that the class of random variables $S_n(f)$, $f\in\Cal F$, is countably approximable. Then the random variables $S_n(f)$, $f\in\Cal F$, satisfy relation~(4.4).} \medskip This corollary can be simply proved, we only have to apply Theorem~4.1 for the class $\Cal F'$. To do this we have to show that if $\Cal F$ is an $L_2$-dense class with some parameter $D$ and exponent $L$, and $\Cal F'\subset \Cal F$, then $\Cal F'$ is also an $L_2$-dense class with the same exponent $L$, only with a possibly different parameter~$D'$. To prove this statement let us choose for all numbers $1\ge\e>0$ and probability measures $\nu$ on $(Y,\Cal Y)$ some functions $f_1,\dots,f_m\in \Cal F$ with $m\le D\(\frac\e2\)^{-L}$ elements, such that the sets $\Cal D_j=\left\{f\:\int |f-f_j|^2\,d\nu\le \(\frac\e2\)^2\right\}$ satisfy the relation $\bigcupp_{j=1}^m \Cal D_j=Y$. For all sets $\Cal D_j$ for which $\Cal D_j\cap \Cal F'$ is non-empty choose a function $f'_j\in \Cal D_j\cap \Cal F'$. In such a way we get a collection of functions $f'_j$ from the class $\Cal F'$ containing at most $2^LD\e^{-L}$ elements which satisfies the condition imposed for $L_2$-dense classes with exponent $L$ and parameter $2^LD$ for this number $\e$ and measure $\nu$. \medskip Given a sequence of independent $\mu$ distributed random variables $\xi_1,\dots,\xi_n$ taking values on $(X,\Cal X)$ let us introduce their empirical distribution on $(X,\Cal X)$ as $$ \mu_n(A)(\oo)=\frac1n \#\left\{j\: 1\le j\le n,\; \xi_j(\oo)\in A\right\}, \quad A\in \Cal X, \tag4.5 $$ and define for all measurable (and integrable) functions $f$ the (random) integral $$ J_n(f)=J_{n,1}(f)=\sqrt n\int f(x)(\mu_n(\,dx)-\mu(\,dx)). \tag4.6 $$ Clearly $J_n(f)=\frac1{\sqrt n}\summ_{j=1}^n (f(\xi_j)-Ef(\xi_j)) =S_n(\bar f)$ with $\bar f(x)=f(x)-\int f(x)\mu(\,dx)$. It is not difficult to see that $\supp_{x\in X}|\bar f(x)|\le2$ if $\supp_{x\in X}|f(x)|\le1$, $\int \bar f(x)\mu(\,dx)=0$, $\int \bar f^2(x)\mu(\,dx)\le\int f^2(x)\mu(\,dx)$, if $\Cal F$ is an $L_2$-dense class of functions with parameter $D$ and exponent $L$, then the class of functions $\bar{\Cal F}$ consisting of the functions $\bar f(x)=f(x)-\int f(x)\mu(\,dx)$, $f\in\Cal F$, is an $L_2$-dense class of functions with parameter $2^LD$ and exponent $L$, since $\int(\bar f-\bar g)^2\,d\mu\le\e$ if $f,g\in\Cal F$, and $\int(f-g)^2\,d\mu\le\(\frac\e2\)^2$. Hence Theorem~4.1 implies the following result which can be considered as its version reformulated for integrals with respect to normalized empirical measures. \medskip\noindent {\bf Theorem 4.1$'$.} {\it Let us have a sequence of iid. random variables $\xi_1,\dots,\xi_n$, $n\ge2$, with distribution $\mu$ on a measurable space $(X,\Cal X)$ together with some class of functions $\Cal F$ on this space which satisfy the conditions of Theorem~4.1 with the possible exception of condition (4.3). Then the estimate (4.4) remains valid if we replace the random sums $S_n(f)$ in it by the random integrals $J_n(f)$ defined in (4.6). Moreover, similarly to the corollary of Theorem 4.1, the countable cardinality of the set $\Cal F$ can be replaced by the condition that the class of random variables $J_n(f)$, $f\in\Cal F$, is countably approximable.} \medskip All finite dimensional distributions of the set of random variables $S_n(f)$, $f\in\Cal F$, converge to a Gaussian field $Z(f)$, $f\in\Cal F$, as $n\to\infty$ with expectation $EZ(f)=0$ and correlation $EZ(f)Z(g)=\int f(x)g(x)\mu(\,dx)$, $f,g\in\Cal F$. (Here and in the subsequent part of the paper a collection of random variables indexed by some set of parameters will be called a Gaussian field if for all finite subsets of these parameters the random variables indexed by this finite set are jointly Gaussian.) Hence we can expect that the random variables of a Gaussian field with such properties satisfy a result similar to Proposition~4.1. The following result can be considered as the Gaussian counterpart of Theorem~4.1. \medskip\noindent {\bf Theorem 4.2.} {\it Let us fix some probability measure $\mu$ on a measurable space $(X,\Cal X)$ together with a countable set $\Cal F$ of square integrable functions with respect to the measure $\mu$ such that there exists a parameter $D>0$ and exponent $L\ge1$ with the following property: For all $\e>0$ there exist $m\le D\e^{-L}$ functions $f_j=f_j(\e)\in\Cal F$, $1\le j\le m$, such that for all $f\in \Cal F$ $\inff_{1\le j\le m}\int(f_j(x)-f(x))^2\mu(\,dx)<\e^2$. Let us also assume that the class of functions $\Cal F$ satisfies condition (4.2) with some $1\ge\sigma>0$. Let us consider a Gaussian field $Z(f)$, $f\in\Cal F$, such that $EZ(f)=0$, $EZ(f)Z(g)=\int f(x)g(x)\mu(\,dx)$, $f,g\in\Cal F$. Then there exist some constants $C>0$ and $M>0$ (for instance $C=4$ and $M=16$ can be chosen) such that the inequality $$ P\(\supp_{f\in\Cal F}|Z(f)|\ge u\)\le C(D+1) \exp\left\{-\frac1{256} \(\frac u{\sigma}\)^2\right\} \quad \text{if }u\ge ML^{1/2}\sigma \log^{1/2}\frac2\sigma \tag4.7 $$ holds with the parameter $D$ and exponent $L$ introduced in this theorem.} \medskip In the inequalities of the above results I did not try to find the best possible universal constants. One could choose for instance the coefficient $\frac{1-\e}2$ with arbitrary small $\e>0$ instead of the coefficient $\frac1{256}$ in the exponent at the right-hand side of formula (4.7) if the other universal constants $C>0$ and $M>0$ are chosen sufficiently large in this inequality. This means that in the bound~(4.7) we can get an estimate with an almost as good exponential term as in the estimate of the probability $P(Z(f)>u)$ for a single Gaussian random variable $Z(f)$ with $EZ(f)=0$, $\Var Z(f)=\sigma^2$. Similarly, the constant $\alpha>0$ can be chosen as $\alpha=\frac{1-\e}2$ with arbitrary small $\e>0$ in formula (4.4). The condition about the countable cardinality of the set $\Cal F$ in Theorem~4.2 could be weakened similarly to Theorem~4.1. But here I omit the discussion of this question, since Theorem~4.2 was only introduced for the sake of a comparison between the Gaussian and non-Gaussian case. An essential difference between Theorems~4.1 and~4.2 is that in Theorem~4.1 the condition was imposed that the class of functions $\Cal F$ has to be $L_2$-dense, while in Theorem~4.2 only a weaker version of this property was needed. In that result we only demanded that there exists a relatively small subset of $\Cal F$ dense in the $L_2(\mu)$ norm. It may demand some explanation why the $L_2$-density property was imposed in Theorem~4.1, a property where also such probability measures $\nu$ are considered which seem to have no relation to the original problem. But as we shall see, the proof of Theorem~4.1 contains a conditioning argument where new conditional measures appear and the $L_2$-density property is needed to work with them. One would also like to know some results which enable us to check when this condition holds. In the next section we shall discuss a popular notion, the notion of Vapnik--\v{C}ervonenkis classes and show that a Vapnik--\v{C}ervonenkis class of functions bounded by~1 is $L_2$-dense. Another difference between Theorems~4.1 and~4.2 is that the conditions of formula (4.4) contain the upper bound $n\sigma^2>\sqrt nu$, and no such condition is imposed in formula (4.7). This difference can be simply explained, since as we have seen in Section~3 in the case $n\sigma^2=\Var(\sqrt n S_n)\ll \sqrt nu$ we can guarantee only a weak non-Gaussian type estimate for the single probabilities $P(\sqrt n S_n(f)>\sqrt nu)$, $f\in\Cal F$. It has a similar reason why condition (4.1) about the supremum of the functions $f\in \Cal F$ appeared in Theorems 4.1 and $4.1'$, and no such condition was needed in Theorem~4.2. The lower bounds for the level~$u$ were imposed in formulas (4.4) and (4.7) because of a similar reason. To understand why such a condition is needed in formula (4.7) let us consider the following example. Take a Wiener process $W(t)$, $0\le t\le1$, define the functions $f_{s,t}(\cdot)$ on the interval $[0,1]$ by the formula $f_{s,t}(u)=1$ if $s\le u\le t$, $f_{s,t}(u)=0$ if $0\le u0$ let us consider the class of functions $\Cal F_\sigma=\{f_{s,t}\: \int f^2_{s,t}(u)\,du=t-s\le\sigma^2, s \text { and }t \text{ are rational numbers}\}$. It is not difficult to see that the above example satisfies the conditions of Theorem~4.2. It is natural to expect that $P\(\supp_{f\in\Cal F_\sigma} Z(f)>u\)\le e^{-\const (u/\sigma)^2}$. However, this relation does not hold if $u=u(\sigma)<(1-\e)\sqrt2\sigma\log^{1/2}\frac1\sigma$ with some $\e>0$. In such cases $P\(\supp_{f\in\Cal F_\sigma}Z(f) >u\)\to1$, as $\sigma\to0$. This can be proved relatively simply with the help of the estimate $P(Z(f_{s,t})>u(\sigma))\ge\const \sigma^{1-\e}$ if $|t-s|=\sigma^2$ and the independence of the random integrals $Z(f_{s,t})$ if the functions $f_{s,t}$ are indexed by such pairs $(s,t)$ for which the intervals $(s,t)$ are disjoint. This means that in this example formula (4.7) holds only under the condition $u\ge M\sigma\log^{1/2}\frac1\sigma$ with $M=\sqrt2$. Some additional work would show that a similar picture arises in the model where we consider the integrals $J_n(f_{s,t})$ of the functions from the same the class $\Cal F_\sigma$ with respect to the normalized empirical measure of a sample of size $n$ with uniform distribution on the interval $[0,1]$ instead of a Wiener process. In this example we have to impose the condition $\sqrt nu\ge M\sqrt n\sigma\log^{1/2}\frac1\sigma$ with $M=\sqrt2$ for the validity of relation (4.4). At a heuristic level it is clear that in the case of a class $\Cal F$ with a large exponent $L$ we have to put a larger coefficient of $\sqrt n\sigma \log^{1/2} \frac2\sigma$ in the condition of formula (4.4) for the validity of Theorem~4.1 or 4.1$'$, and a similar statement can be told about the condition (4.7) in Theorem~4.2. (I did not try to find the best possible coefficients in the conditions of relations (4.4) and (4.7), they could be improved considerably.) In Theorem~4.1 (and in its version 4.1$'$) it was demanded that the class of functions $\Cal F$ should be countable. Later this condition was replaced by a weaker condition about countable approximability. By restricting our attention to countable or countably approximable classes we could avoid some unpleasant measure theoretical problems which would have arisen if we had worked with the supremum of non-countable number of random variables which may be non-measurable. There are some papers where possibly non-measurable models are also considered with the help of some rather deep results of the analysis and measure theory. Actually, the problem we met here is the natural analog of an important problem in the theory of the stochastic processes about the smoothness property of the trajectories of an appropriate version of a stochastic process which we can get by exploiting our freedom to change all random variables on a set of probability zero. The study of the problem in this work is simpler in one respect. Here the set of random variables $S_n(f)(\oo)$ or $J_n(f)(\oo)$, $f\in\Cal F$, are constructed directly with the help of the underlying random variables $\xi_1(\oo),\dots,\xi_n(\oo)$ for all $\oo\in\Cal\Omega$ separately. We are interested in when the sets of random variables constructed in this way are countably approximable, i.e.\ we are not looking for a possibly different, better version of them with the same finite dimensional distributions. In the next simple Lemma~4.3 we give a sufficient condition for countable approximability. Its condition can be interpreted as a smoothness type condition for the trajectories of a stochastic process indexed by the functions $f\in\Cal F$. \medskip\noindent {\bf Lemma 4.3.} {\it Let a class of random variables $U(f)$, $f\in\Cal F$, indexed by some set $\Cal F$ of functions on a space $(Y,\Cal Y)$ be given. If there exists a countable subset $\Cal F'\subset \Cal F$ of the set $\Cal F$ such that the sets $A(u)=\{\oo\:\supp_{f\in \Cal F}|U(f)(\oo)|\ge u\}$ and $B(u)=\{\oo\:\supp_{f\in \Cal F'} |U(f)(\oo)|\ge u\}$ introduced for all $u>0$ in the definition of countable approximability satisfy the relation $A(u)\subset B(u-\e)$ for all $u>\e>0$, then the class of random variables $U(f)$, $f\in\Cal F$, is countably approximable. The above property holds if for all $f\in\Cal F$, $\e>0$ and $\oo\in\Omega$ there exists a function $\bar f=\bar f(f,\e,\oo) \in\Cal F'$ such that $|U(\bar f)(\oo)|\ge|U(f)(\oo)|-\e$.} \medskip\noindent {\it Proof of Lemma 4.3.}\/ If $A(u)\subset B(u-\e)$ for all $\e>0$, then $P^*(A(U)\setminus B(u))\le \limm_{\e\to0} P(B(u-\e)\setminus B(u))=0$, where $P^*(X)$ denotes the outer measure of a not necessarily measurable set $X\subset\Omega$, since $\bigcapp_{\e\to0}B(u-\e)=B(u)$, and this is what we had to prove. If $\oo\in A(u)$, then for all $\e>0$ there exists some $f=f(\oo)\in\Cal F$ such that $|U(f)(\oo)|>u-\frac\e2$. If there exists some $\bar f=\bar f(f,\frac\e2,\oo)$, $f\in\Cal F'$ such that $|U(\bar f)(\oo)| \ge |Uf(\oo)|-\frac\e2$, then $|U(\bar f)(\oo)| >u-\e$, and $\oo\in B(u-\e)$. This means that $A(u)\subset B(u-\e)$. \medskip The question about countable approximability also appears in the case of multiple random integrals. To avoid some repetition we prove a result which also covers such cases. For this goal first we introduce the notion of multiple integrals with respect to a normalized empirical measure. Given a measurable function $f(x_1,\dots,x_k)$ on the $k$-fold product space $(X^k,\Cal X^k)$ and a sequence of independent random variables $\xi_1,\dots,\xi_n$ with some distribution $\mu$ on the space $(X,\Cal X)$ define the integral $J_{n,k}(f)$ of the function $f$ with respect to the $k$-fold product of the normalized empirical measure $\mu_n$ introduced in (4.5) by the formula $$ \allowdisplaybreaks \align J_{n,k}(f)&=\dfrac{n^{k/2}}{k!} \int' f(x_1,\dots,x_k)(\mu_n(\,dx_1)-\mu(\,dx_1))\dots (\mu_n(\,dx_k)-\mu(\,dx_k)),\\ &\qquad\text{where the prime in $\tsize\int'$ means that the diagonals } x_j=x_l,\; 1\le j\e>0$ and $\oo\in\Omega$ let us choose a function $f_{\bar u_1, \dots, \bar u_k,\bar v_1,\dots,\bar v_k}\in \Cal F'$ determined with some vectors $\bar u_j=\bar u_j(\e,\oo)$, $\bar v_j=\bar v_j(\e,\oo)$ $1\le j\le k$, with rational coordinates $\bar u_j\le u_ju\)$. We have seen that the above class of functions $\Cal F$ is countably approximable. The results of the next section imply that this class of functions is also $L_2$-dense. Otherwise it is not difficult to check this property directly. Hence we can apply Theorem~4.1 to the above defined class of functions with $\sigma=1$, and it yields that $P\(n^{-1/2}\supp_{f\in \Cal F}|J_n(f)|>u\)\le e^{-Cnu^2}$ if $1\ge u\ge\bar Cn^{-1/2}$ with some universal constants $C>0$ and $\bar C>0$. (The condition $1\ge u$ can actually be dropped.) The application of this estimate for the numbers $\e>0$ together with the Borel-Cantelli lemma imply the fundamental theorem of the mathematical statistics. In short, the results of this section yield more information about the closeness the empirical distribution function $F_n$ and distribution function $F$ than the fundamental theorem of the mathematical statistics. Moreover, since these results can also be applied for other classes of functions they yield useful information about the closeness of the probability measure $\mu$ and empirical measure~$\mu_n$. \beginsection 5. Vapnik--\v{C}ervonenkis classes and $L_2$-dense classes of functions In this section the most important notions and results will be presented about Vapnik--\v{C}ervonenkis classes, and it will be explained how they help to show in some important cases that certain classes of functions are $L_2$-dense. Some proofs are put in the Appendix. First I recall the following notions. \medskip\noindent {\bf Definition of Vapnik-\v{C}ervonenkis classes of sets and functions.} {\it Let a set $S$ be given, and let us select a class $\Cal D$ consisting of certain subsets of this set $S$. We call $\Cal D$ a Vapnik--\v{C}ervonenkis class if there exist two real numbers $B$ and $K$ such that for all positive integers $n$ and subsets $S_0(n)=\{x_1,\dots,x_n\}\subset S$ of cardinality $n$ of the set $S$ the collection of sets of the form $S_0(n)\cap D$, $D\in\Cal D$, contains no more than $Bn^K$ subsets of~$S_0(n)$. We shall call $B$ the parameter and $K$ the exponent of this Vapnik--\v{C}ervonenkis class. A class of real valued functions $\Cal F$ on a space $(Y,\Cal Y)$ is called a Vapnik--\v{C}ervonenkis class if the collection of graphs of these functions is a Vapnik--\v{C}ervonenkis class, i.e.\ if the sets $A(f)=\{(y,t)\: y\in Y,\;\min(0,f(y))\le t\le \max(0,f(y))\}$, $f\in \Cal F$, constitute a Vapnik--\v{C}er\-vo\-nen\-kis class of subsets of the product space $S=Y\times R^1$.} \medskip The following result which was first proved by Sauer is of fundamental importance in the theory of Vapnik--\v{C}er\-vo\-nen\-kis classes. Its proof is given in the Appendix. \medskip\noindent {\bf Theorem 5.1 (Sauer's lemma).} {\it Let a set $S$ be given together with a class $\Cal D$ of subsets of this set $S$. Fix some subset $S_0=S_0(n)$ of the set $S$ containing $n$ point and consider the class of subsets $\Cal D(S_0)=\{S_0\cap D\: D\in\Cal D\}$ of $S_0$ consisting of the intersections of the set $S_0$ with the elements of the class $\Cal D$. If there is some positive integer $k$ such that all subsets $F\subset S_0$ of cardinality $k$ have at least one ``hidden'' subset not contained in the collection of sets $\Cal D(S_0,F)=\{F\cap B;\,B\in \Cal D(S_0)\}$, then $\Cal D(S_0)$ contains at most $\binom n0+\binom n1+\cdots+\binom n{k-1}$ subsets of $S$.} \medskip Theorem 5.1 has the remarkable consequence that if there exists some integer $k$ such that for all subsets $S_0(k)$ of cardinality $k$ of the set $S$ the number of sets of the form $S_0(k)\cap D$, $D\in\Cal D$, is less than $2^k$, (i.e. not all subsets of $S_0(k)$ can be represented in this form,) then $S_0(n)\cap \Cal D$ has at most $\binom n0+\binom n1+\cdots+\binom n{k-1}$ elements for all subsets $S_0(n)$ of the set $S$ with $n\ge k$ elements, since in this case the conditions of Theorem~5.1 hold for all $n\ge k$ and subset $S_0(n)\subset S$ of $S$ of cardinality~$n$ and this number~$k$. This means that in this case $\Cal D$ is a Vapnik-\v{C}ervonenkis class. It can be proved that $\binom n0+\binom n1+\cdots+\binom n{k-1} \le1.5\frac{n^{k-1}}{(k-1)!}$ if $n\ge k+1$, and this relation enables us to give an explicit estimate on the exponent and parameter of this Vapnik--\v{C}ervonenkis class. Hence we have to check a seemingly much weaker property to show that a class of subsets of a set $S$ is a Vapnik--\v{C}ervonenkis class. Moreover, Theorem~5.1 implies that there are two cases. Either there is some set $S_0(n)$ of cardinality $n$ for all integers $n$ such that $\Cal D(S_0(n))$ contains all subsets of $S_0(n)$ or $\supp_{S_0\subset S,|S_0|=n}|\Cal D(S_0)|$ tends to infinity in polynomial order as $n\to\infty$, where $|S_0|$ and $|\Cal D(S_0)|$ denotes the cardinality of $S_0$ and $\Cal D(S_0)$. The upper bound given for $|\Cal D(S_0)|$ in Theorem~5.1 appears in a natural way. If $\Cal D(S_0)$ consists of the subsets of $S_0$ of cardinality less than or equal to $k-1$, then the above sum equals $|\Cal D(S_0)|$. In such a case the conditions of Theorem~5.1 are satisfied, and the proof of Theorem~5.1 shows that this is the extreme case, this is the largest class of sets $S_0(n)\cap \Cal D$ satisfying Theorem~5.1. The following Theorem~5.2, an important result of Richard Dudley, states that a Vapnik--\v{C}ervonenkis class of functions bounded by~1 is an $L_1$-dense class of functions. \medskip\noindent {\bf Theorem 5.2.} {\it Let $f(y)$, $f\in \Cal F$, be a Vapnik--\v{C}ervonenkis class of real valued functions on some measurable space $(Y,\Cal Y)$ such that $\supp_{y\in Y}|f(y)|\le1$ for all $f\in\Cal F$. Then $\Cal F$ is an $L_1$-dense class of functions on $(Y,\Cal Y)$. More explicitly, if $\Cal F$ is a Vapnik--\v{C}ervonenkis class with parameter $B\ge1$ and exponent $K>0$, then it is an $L_1$-dense class with exponent $L=2K$ and parameter $D=CB^2 (4K)^{2K}$ with some universal constant $C>0$.} \medskip\noindent {\it Proof of Theorem 5.2.}\/ Let us fix some probability measure $\nu$ on $(Y,\Cal Y)$ and a real number $1\ge\e>0$. We are going to show that the cardinality of any finite set $\Cal D(\e,\nu)=\{f_1,\dots,f_M\} \subset \Cal F$ such that $\int|f_j-f_k|\,d\nu\ge\e$ if $j\neq k$, $f_j,f_k\in \Cal D(\e,\nu)$ has a cardinality $M\le D\e^{-L}$ with some $D>0$ and $L>0$. This implies that $\Cal F$ is an $L_1$-dense class with parameter~$D$ and exponent~$L$. Indeed, let us take a maximal subset $\bar{\Cal D}(\e,\nu)=\{f_1,\dots,f_M\}$ such that the $L_1(\nu)$ distance of any two functions in this subset is at least~$\e$. Maximality means in this context that no function $f_{M+1}$ can be attached to $\bar{\Cal D}(\e,\nu)$ without violating this condition. If we show that $M\le D\e^{-L}$, then this means that $\bar{\Cal D}(\e,\nu)$ is an $\e$-dense subset of~$\Cal F$ in the space $L_p(Y,\Cal Y,\nu)$ with no more than $D\e^{-L}$ elements. In the estimation of the cardinality $M$ of $\Cal D(\e,\nu)$ we exploit the Vapnik--\v{C}ervonenkis class property of $\Cal F$ in the following way. Let us choose relatively few $p$ points $(y_l,t_l)$, $y_l\in Y$, $-1\le t_l\le1$, $1\le l\le p$, in the space $(Y\times [-1,1])$ in such a way that the set $S_0(p)=\{(y_l,t_l),\,1\le l\le p\}$ and graphs $A(f_j)=\{(y,t)\: y\in Y,\;\min(0,f_j(y))\le t\le \max(0,f_j(y))\}$, $f\in \Cal F$, $f_j\in\Cal D(\e,\nu)$ have the property that all sets $A(f_j)\cap S_0(p)$, $1\le j\le M$, are different. Then the Vapnik--\v{C}ervonenkis class property of $\Cal F$ implies that $M\le Bp^K$. Hence if we can construct a set $S_0(p)$ with the above property with a relatively small number $p$, then we get a useful estimate on $M$. Such a set $S_0(p)$ will be given by means of the following random construction. Let us choose the $p$ points $(y_l,t_l)$, $1\le l\le p$, of the (random) set $S_0(p)$ independently of each other in such a way that the coordinate $y_l$ is chosen with distribution $\nu$ on $(Y,\Cal Y)$ and the coordinate $t_l$ with uniform distribution on the interval $[-1,1]$ independently of $y_l$. (The number~$p$ will be chosen later.) Let us fix some indices $1\le j,k\le M$ and estimate the probability that the sets $A(f_j)\cap S_0(p)$ and $A(f_k)\cap S_0(p)$ agree, where $A(f)$ denotes the graph of the function~$f$. Consider the symmetric difference $A(f_j)\Delta A(f_k)$ of the sets $A(f_j)$ and $A(f_k)$. The sets $A(f_j)\cap S_0(p)$ and $A(f_k)\cap S_0(p)$ agree if and only if $(y_l,t_l)\notin A(f_j)\Delta A(f_k)$ for all $(y_l,t_l)\in S_0(p)$. Let us observe that for a fixed $l$ $P((y_l,t_l)\in A(f_j)\Delta A(f_k))=\frac12(\nu\times\lambda) (A(f_j)\Delta A(f_k))=\frac12\int |f_j-f_k|\,d\nu\ge\frac\e2$, where $\lambda$ denotes the Lebesgue measure. This implies that the probability that $A(f_j)\cap S_0(p)$ and $A(f_k)\cap S_0(p)$ agree can be bounded from above by $\(1-\frac\e2\)^p\le e^{-p\e/2}$. Hence the probability that all sets $A(f_j)\cap S_0(p)$ are different is greater than $1-\binom M2 e^{-p\e/2}\ge1-\frac{M^2}2e^{-p\e/2}$. Choose $p$ such that $\frac74e^{p\e/2}>e^{(p+1)\e/2}>M^2\ge e^{p\e/2}$. Then the above probability is greater than $\frac18$, and there exists some set $S_0(p)$ with the desired property. The inequalities $M\le Bp^K$ and $M^2\ge e^{p\e/2}$ imply that $M\ge e^{\e M^{1/K}/4B^{1/K}}$, i.e.\ $\frac{\log M^{1/K}}{M^{1/K}} \ge \frac\e{4KB^{1/K}}$. As $\frac{\log M^{1/K}}{M^{1/K}}\le CM^{-1/2K}$ for $M\ge1$ with some universal constant $C>0$, this estimate implies that Theorem 5.2 holds with the exponent $L$ and parameter $D$ given in its formulation. \medskip Let us observe that if $\Cal F$ is an $L_1$-dense class of functions on a measure space $(Y,\Cal Y)$ with some exponent $L$ and parameter $D$, and also the inequality $\supp_{y\in Y}|f(y)|\le1$ holds for all $f\in\Cal F$, then $\Cal F$ is an $L_2$-dense class of functions with exponent $2L$ and parameter $D2^L$. Indeed, if we fix some measure $\nu$ on $(Y,\Cal Y)$ together with a number $1\ge\e>0$, and $\Cal D(\e,\nu)=\{f_1,\dots, f_M\}$ is an $\frac{\e^2}2$-dense set of $\Cal F$ in the space $L_1(Y,\Cal Y,\nu)$, $M\le2^L D \e^{-2L}$ then for any $f\in \Cal F$ we can choose some $f_j\in\Cal D(\e,\nu)$ such that $\int(f-f_j)^2\,d\nu\le2\int|f-f_j|\,d\nu\le\e^2$. This means that $\Cal F$ is really an $L_2$-dense class with the given exponent and parameter. It is not easy to check whether a collection of subsets $\Cal D$ of a set $S$ is a Vapnik--\v{C}ervonenkis class even with the help of Theorem~5.1. Therefore the following Theorem~5.3 which enables us to construct many non-trivial Vapnik--\v{C}ervonenkis classes is of special interest. Its proof is also put in the Appendix. \medskip\noindent {\bf Theorem 5.3.} {\it Let us consider a $k$-dimensional subspace $\Cal G_k$ of the linear space of real valued functions defined on a set $S$, and define the level-set $A(g)=\{s\:s\in S,\, g(s)\ge0\}$ for all functions $g\in\Cal G_k$. Take the class of subsets $\Cal D=\{A(g)\: g\in \Cal G_k\}$ of the set $S$ consisting of the above introduced level sets. All subsets $S_0=S_0(k+1)\subset S$ of cardinality $k+1$ has a ``hidden'' subset which is not contained in the class of subsets $\Cal D(S_0)=\{S_0\cap D\: D\in\Cal D\}$ of $S_0$ introduced in Theorem~5.1. By Theorem~5.1 this property implies that the class of sets $\Cal D$ is a Vapnik--\v{C}ervonenkis class.} \medskip Theorem~5.3 enables us to construct many interesting Vapnik--\v{C}ervonenkis classes. Thus for instance the class of all half-spaces in a Euclidean space, the class of all ellipses in the plane, or more generally the level sets of $k$-order algebraic functions with a fixed number $k$ constitute a Vapnik--\v{C}ervonenkis class. It can be proved that if $\Cal C$ and $\Cal D$ are Vapnik--\v{C}ervonenkis classes of subsets of a set $S$, then also their intersection $\Cal C\cap \Cal D=\{C\cap D\:C\in\Cal C,\,D\in\Cal D\}$, their union $\Cal C\cup \Cal D=\{C\cup D\:C\in\Cal C,\,D\in\Cal D\}$ and complementers $\Cal C^c=\{S\setminus C\:C\in\Cal C\}$ are Vapnik--\v{C}ervonenkis classes. These results are less important for us and their proofs will be omitted. We are interested in Vapnik--\v{C}ervonenkis classes not for their own sake. We are going to study $L_2$-dense classes of functions, and Vapnik--\v{C}ervonenkis classes make possible to find some examples. Indeed, Theorem 5.2 implies that if $\Cal D$ is a Vapnik--\v{C}ervonenkis class of subsets of a set $S$, then their indicator functions constitute an $L_1$-dense, hence also an $L_2$-dense class of functions. Then the results of Lemma~5.4 formulated below enable us to construct new $L_2$-dense class of functions. The description of $L_2$-dense classes of functions are interesting for us, because they appear in the conditions of the results in Section~4. \medskip\noindent {\bf Lemma 5.4.} {\it Let $\Cal G$ be an $L_2$-dense class of functions on some space $(Y,\Cal Y)$ whose absolute values are bounded by one, and let $f$ be a function on $(Y,\Cal Y)$ also with absolute value bounded by one. Then $f\cdot\Cal G=\{f\cdot g\: g\in G\}$ is also an $L_2$-dense class of functions. Let $\Cal G_1$ and $\Cal G_2$ be two $L_2$-dense classes of functions on some space $(Y,\Cal Y)$ whose absolute values are bounded by one. Then the classes of functions $\Cal G_1+\Cal G_2=\{g_1+g_2\:g_1\in\Cal G_1,\,g_2\in\Cal G_2\}$, $\Cal G_1\cdot\Cal G_2=\{g_1g_2\:g_1\in\Cal G_1,\,g_2\in\Cal G_2\}$, $\min(\Cal G_1,\Cal G_2)=\{\min(g_1,g_2)\:g_1\in\Cal G_1,\,g_2\in\Cal G_2\}$, $\max(\Cal G_1,\Cal G_2)=\{\max(g_1,g_2)\:g_1\in\Cal G_1,\,g_2\in\Cal G_2\}$ are also $L_2$-dense. If $\Cal G$ is an $L_2$-dense class of functions, and $\Cal G'\subset G$, then $\Cal G'$ is also an $L_2$-dense class.} \medskip\noindent The proof of Lemma 5.4 is rather straightforward. One has to observe for instance that if $g_1,\bar g_1\in\Cal G_1$, $g_2,\bar g_2\in\Cal G_2$ then $|\min(g_1,g_2)-\min(\bar g_1,\bar g_2)|\le |g_1-\bar g_1)|+|g_2-\bar g_2|$, hence if $g_{1,1},\dots,g_{1,M_1}$ is an $\frac\e2$-dense subset of $\Cal G_1$ and $g_{2,1},\dots,g_{2,M_2}$ is an $\frac\e2$-dense subset of $\Cal G_2$ in the space $L_2(Y,\Cal Y,\nu)$ with some probability measure $\nu$, then the functions $\min(g_{1,j},g_{2,k})$, $1\le j\le M_1$, $1\le k\le M_2$ constitute an $\e$-dense subset of $\min(\Cal G_1,\Cal G_2)$ in $L_2(Y,\Cal Y,\nu)$. The last statement of Lemma 5.4 is proved after the Corollary of Theorem~4.1. The details are left to the reader. The above results enable us to find some interesting classes of $L_2$-dense classes of functions. In particular, the indicator functions of Vapnik-\v{C}ervonenkis class of sets is an $L_2$-dense class of functions, and then Lemma~5.4 enables us to construct new classes of $L_2$-dense classes of functions with their help. It is not difficult to see with the help of these results for instance that the random variables considered in Lemma 4.4 are not only countably approximable, but the class of functions $f_{u_1,\dots,u_k,v_1,\dots,v_k}$ taking part in their definition is $L_2$-dense. \beginsection 6. The proof of Theorems 4.1 and 4.2 on the supremum of random sums This section contains the proof of some results which can be proved by means of a simple but useful method, the so-called chaining argument. This method enables us to prove Theorem~4.2 completely, but it only helps to reduce Theorem~4.1 to a slightly simpler statement presented in Proposition~6.1. We also formulate another result in Proposition~6.2 and show that these two propositions together imply Theorem~4.1. The proof of Proposition 6.2 which is based on a symmetrization argument is left to the next section. The method of proof of Theorem~4.2 does not suffice in itself to prove Theorem~4.1, because we have relatively weak estimates about the distribution of sums of independent random variables with small variances. This does not allow to follow the chaining argument in the proof of Theorem~4.1 up to the end, we have to stop at a point. In such a way we only get a seemingly weak result, but as it turns out this is the result we need to cover that part of Theorem~4.1 which cannot be handled by means of the symmetrization method applied in the proof of Proposition~6.2. First we prove Theorem~4.2. \medskip\noindent {\it Proof of Theorem 4.2.}\/ Let us list the elements of $\Cal F$ as $\{f_0,f_1,\dots\}=\Cal F$, and choose for all $p=0,1,2,\dots$ a set of functions $\Cal F_p=\{f_{a(p,1)},\dots,f_{a(p,m_p)}\}\subset\Cal F$ with $m_p\le (D+1)\,2^{2pL}\sigma^{-L}$ elements in such a way that $\inff_{1\le j\le m_p} \int (f-f_{a(p,j)})^2\,d\mu\le 2^{-4p}\sigma^2$ for all $f\in\Cal F$, and $f_p\in \Cal F_p$. For all indices $a(p,j)$ of the functions in $\Cal F_p$, $p=1,2,\dots$, define a predecessor $a(p-1,j')$ from the indices of the set of functions $\Cal F_{p-1}$ in such a way that the functions $f_{a(p,j)}$ and $f_{a(p-1),j')}$ satisfy the relation $\int(f_{(p,j)}-f_{(p-1,j')})^2\,d\mu\le2^{-4(p-1)}\sigma^2$. With the help of the behaviour of the standard normal distribution function we can write the estimates $$ \align P(A(p,j))&=P\(|Z(f_{a(p,j)})-Z(f_{a(p-1,j')})|\ge 2^{-(1+p)}u\) \le 2\exp\left\{-\frac{2^{-2(p+1)}u^2}{2\cdot 2^{-4(p-1)}\sigma^2} \right\}\\ &=2\exp\left\{-\frac{2^{2p}u^2}{128\sigma^2}\right\} \quad 1\le j\le m_p,\; p=1,2,\dots, \endalign $$ and $$ P(B(j))=P\(|Z(f_{a(0,j)})|\ge \frac u2\)\le \exp\left\{-\frac {u^2}{8\sigma^2}\right\}, \quad 1\le j\le m_0. $$ The above estimates together with the relation $\bigcupp_{p=0}^\infty \Cal F_p=\Cal F$ which implies that \hfill\break $\{|Z(f)|\ge u\}\subset\bigcupp_{p=1}^\infty\bigcupp_{j=1}^{m_p}A(p,j) \cup\bigcupp_{s=1}^{m_0}B(s)$ for all $f\in\Cal F$ yield that $$ \align &P\(\sup_{f\in\Cal F} |Z(f)|\ge u\) \le P\(\bigcup_{p=1}^\infty\bigcup_{j=1}^{m_p}A(p,j) \cup\bigcup_{s=1}^{m_0}B(s)\) \\ &\qquad \le \sum_{p=1}^\infty\sum_{j=1}^{m_p}P(A(p,j)) +\sum_{s=1}^{m_0}P(B(s)) \\ &\qquad \le \sum_{p=1}^{\infty} 2(D+1)2^{2pL} \sigma^{-L} \exp\left\{-\frac{2^{2p}u^2}{128\sigma^2} \right\} +2(D+1)\sigma^{-L} \exp\left\{-\frac {u^2}{8\sigma^2}\right\}. \endalign $$ If $u\ge ML^{1/2}\sigma\log \frac2\sigma$ with $M\ge16$ (and $L\ge1$), then $$ 2^{2pL}\sigma^{-L}\exp\left\{-\frac{2^{2p}u^2}{256\sigma^2} \right\}\le\(\frac12\)^{-2pL}\sigma^{-L}\(\frac\sigma 2\)^{2^{2p}M^2L/256}\le 2^{-pL}\le2^{-p} $$ for all $p=0,1\dots$, hence the previous inequality implies that $$ P\(\sup_{f\in\Cal F}|Z(f)|\ge u\)\le 2(D+1)\summ_{p=0}^\infty 2^{-p} \exp\left\{-\frac{2^{2p}u^2}{256\sigma^2} \right\}=4(D+1) \exp\left\{-\frac{u^2}{256\sigma^2} \right\}. $$ Theorem 4.2 is proved. \medskip With the appropriate choice of the bound of the integrals in the definition of the sets $\Cal F_p$ in the proof of Theorem~4.2 and some more calculation it can be proved that the coefficient $\frac1{256}$ in the exponent of the right-hand side (4.7) can be replaced by $\frac{1-\e}2$ with arbitrary small $\e>0$ if the remaining constants in this estimate are chosen sufficiently large. The proof of Theorem 4.2 was based on the fact that sufficiently good estimates can be given on the probabilities $P(|Z(f)-Z(g)|>u)$ for all $f,g\in\Cal F$ and $u>0$. In the case of Theorem 4.1 we only have a weaker estimate for the corresponding probabilities, we cannot give a good estimate on the distribution of the difference $S_n(f)-S_n(g)$ if its variance is small. As a consequence the chaining argument supplies only a weaker result in this case. This result will be given in Proposition~6.1, where the supremum of the normalized random sums $S_n(f)$ is estimated on a relatively dense subset of the class of functions $f\in\Cal F$ in the $L_2(\mu)$ norm. We present another result in Proposition~6.2 which will be proved in the next section and show that Theorem~4.1 follows from these two results. Before the formulation of Proposition~6.1 I recall an estimate which is a simple consequence of Bernstein's inequality: If $S_n(f)=\frac1{\sqrt n}\summ_{j=1}^n f(\xi_j)$ is the normalized sum of independent, identically random variables, $P(|f(\xi_1)|\le1)=1$, $Ef(\xi_1)=0$, $Ef(\xi_1)^2\le\sigma^2$, then there exists some constant $\alpha>0$ such that $$ P(|S_n(f)|>u)\le 2e^{-\alpha u^2/\sigma^2}\quad \text{if}\quad 00$ for which $n\sigma^2\ge \(\frac u\sigma\)^2 \ge ML\log\frac2\sigma$ a number $\bar\sigma=\bar\sigma(u)$, $0\le\bar\sigma\le \sigma\le1$, and a collection of functions $\Cal F_{\bar\sigma}=\{f_1,\dots,f_m\}\subset \Cal F$ with $m\le D\bar\sigma^{-L}$ elements can be chosen in such a way that the sets $\Cal D_j=\{f\:f\in \Cal F,\int|f-f_j|^2\,d\mu \le\bar\sigma^2\}$, $1\le j\le m$, satisfy the relation $\bigcupp_{j=1}^m\Cal D_j=\Cal F$, and the normalized partial sums $S_n(f)$, $f\in\Cal F_{\bar\sigma}$, $n\ge2$, satisfy the inequality $$ P\(\sup_{f\in\Cal F_{\bar\sigma}} |S_n(f)|\ge\frac u{\bar A}\) \le 4D\exp\left\{-\alpha\(\frac u{10\bar A\sigma}\)^2\right\} \quad \text{if}\quad n\sigma^2\ge \(\frac u\sigma\)^2 \ge ML\log\frac2\sigma \tag6.2 $$ with the constants $\alpha$ in formula (6.1) and the exponent $L$ and parameter $D$ of the $L_2$-dense class $\Cal F$. Beside this, also the inequalities $\frac14\(\frac u{\bar A\bar\sigma}\)^2\ge n\bar\sigma^2 \ge\frac1{64}\(\frac u{\bar A\sigma}\)^2$ and $n\bar\sigma^2\ge\frac{M^{2/3}(L+\beta)\log n}{1000\bar A^{4/3}}$ hold with $\beta=\max\(\frac{\log D}{\log n},0\)$, provided that also the inequality $n\sigma^2\ge \(\frac u\sigma\)^2\ge M(L+\beta)^{3/2}\log\frac2\sigma$ holds. (We may assume that the sample size~$n$ is sufficiently large, so the set of numbers~$u$ for which $n\sigma^2\ge \(\frac u\sigma\)^2\ge M(L+\beta)^{3/2} \log\frac2\sigma$ is non-empty.)} \medskip Proposition~6.1 helps to reduce the proof of Theorem~4.1 to the case when the $L_2$~norm of the functions in the class $\Cal F$ is bounded by a relatively small number $\bar\sigma$. In more detail, the proof of Theorem~4.1 can be reduced to a good estimate on the distribution of the supremum of random variables $\supp_{f\in D_j}|S_n(f-f_j)|$ for all classes $\Cal D_j$, $1\le j\le m$, by means of Proposition~6.1. We also have to know that the number~$m$ of the classes $\Cal D_j$ is not too large, otherwise our estimates cannot be useful. A result formulated in Proposition~6.2 helps us to complete the proof of Theorem~4.1. It contains some parameters, and we have to fit the constants in the estimates of Propositions~6.1 and~6.2. This was the reason to introduce the rather artificial parameter~$\bar A\ge2$ in Proposition~6.1 and to formulate the conditions of inequality~(6.2) with a number $M\ge M_0(\bar A)$ instead of a number $M_0$. We want such a formulation of Proposition~6.1 in which it can achieved for any fixed number $A>0$ that the relation $n\bar\sigma^2\ge A\log n$ holds, where the number $\bar\sigma$ was defined in the proof of Proposition~6.1. The last two relations in Proposition~6.1 shows that this is possible if first the number $\bar A$ and then the number $M\ge M_0(\bar A)$ is chosen sufficiently large. Now we formulate Proposition 6.2 and prove Theorem 4.1 with its help. \medskip\noindent {\bf Proposition 6.2.} {\it Let us have a probability measure $\mu$ on a measurable space $(X,\Cal X)$ together with a sequence of independent and $\mu$ distributed random variables $\xi_1,\dots,\xi_n$, $n\ge2$, and a countable, $L_2$-dense class of functions $f=f(x)$ on $(X,\Cal X)$ with some parameter $D$ and exponent $L\ge1$ which satisfies conditions (4.1), (4.2) and (4.3) with some $\sigma>0$ such that the inequality $n\sigma^2>K(L+\beta) \log n$ holds with an appropriate, sufficiently large universal number $K\ge3$ and $\beta=\max\(0,\frac {\log D}{\log n}\)$. Then there exists some universal constant $\gamma>0$ and threshold index $A_0>0$ such that the random sums $S_n(f)$, $f\in \Cal F$, introduced in Theorem~4.1 satisfy the inequality $$ P\(\sup_{f\in\Cal F}|S_n(f)|\ge A n^{1/2}\sigma^2\)\le e^{-\gamma A^{1/2}n\sigma^2}\quad \text{if } A\ge A_0. \tag6.3 $$ (A possible choice of the parameters is: $K=4$, $A_0=2^{10}\cdot10^{16}$ and $\gamma=\frac12$.)} \medskip I did not try to find optimal parameters in formula (6.3). Even the exponent $\frac12$ of~$A$ in the exponent at its right-hand side could be improved. The result of Proposition~6.2 is similar to that of Theorem~4.1. Both of them give an estimate on a probability of the type $P\(\supp_{f\in\Cal F}|S_n(f)|\ge u\)$. The essential difference between them is that in Theorem 4.1 this probability is considered for $u\le\const n^{1/2}\sigma^2$ while in Proposition~6.2 the case $u>\const n^{1/2}\sigma^2$ is looked at. Let us observe that this is the case when no good Gaussian type estimate can be given for the probabilities $P(S_n(f)\ge u)$, $f\in\Cal F$. In this case Bernstein's inequality yields the bound $P(S_n(f)>An^{1/2}\sigma^2)=P\(\summ_{l=1}^nf(\xi_l)>xV_n\)\alpha+u\)\le\frac1\beta P\(\supp_{1\le n<\infty}|Z_n-\bar Z_n|>u\)\quad \text{for all } u>0. $$ }\medskip\noindent {\it Proof of Lemma 7.1.}\/ Put $\tau=\min\{n\: |Z_n|>\alpha+u\}$ if there exists such an index $n$, and $\tau=0$ otherwise. Then the event $\{\tau=n\}$ is independent of the sequence of random variables $\bar Z_1,\bar Z_2,\dots$ for all $n=1,2,\dots$, and because of this independence $$ P(\{\tau=n\})\le\frac1\beta P(\{\tau=n\}\cap\{|\bar Z_n|\le\alpha\}) \le \frac1\beta P(\{\tau=n\}\cap\{|Z_n-\bar Z_n|>u\}) $$ for all $n=1,2,\dots$. Hence $$ \align &P\(\sup_{1\le n<\infty}|Z_n|>\alpha+u\) =\sum_{n=1}^\infty P(\tau=n)\le \frac1\beta \sum_{n=1}^\infty P(\{\tau=n\}\cap\{|Z_n-\bar Z_n|>u\}) \\ &\qquad \le\frac1\beta P\(\supp_{1\le n<\infty}|Z_n-\bar Z_n|>u\). \endalign $$ Lemma 7.1 is proved. \medskip We shall apply the following consequence Lemma~7.2 of the symmetrization lemma. \medskip\noindent {\bf Lemma 7.2.} {\it Let us fix a countable class of functions $\Cal F$ on a measurable space $(X,\Cal X)$ together with a real number $0<\sigma<1$. Consider a sequence of independent, identically distributed $X$-valued random variables $\xi_1,\dots,\xi_n$ such that $Ef(\xi_1)=0$, $Ef^2(\xi_1)\le\sigma^2$ for all $f\in\Cal F$ together with another sequence $\e_1,\dots,\e_n$ of independent random variables with distribution $P(\e_j=1)=P(\e_j=-1)=\frac12$, $1\le j\le n$, independent also of the random sequence $\xi_1,\dots,\xi_n$. Then $$ \aligned &P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n f(\xi_j)\right| \ge A n^{1/2}\sigma^{2}\) \\ &\qquad \le 4P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n \e_jf(\xi_j)\right| \ge \frac A3 n^{1/2}\sigma^{2}\) \quad\text{if } A\ge \frac{3\sqrt2}{\sqrt n\sigma}. \endaligned \tag7.2 $$ }\medskip\noindent {\it Proof of Lemma 7.2.}\/ Let us construct an independent copy $\bar\xi_1,\dots,\bar\xi_n$ of the sequence $\xi_1,\dots,\xi_n$ in such a way that all three sequences $\xi_1,\dots,\xi_n$, \ $\bar\xi_1,\dots,\bar\xi_n$ and $\e_1,\dots,\e_n$ are independent. Define the random variables $S_n(f)=\frac1{\sqrt n}\summ_{j=1}^n f(\xi_j)$ and $\bar S_n(f)=\frac1{\sqrt n}\summ_{j=1}^n f(\bar\xi_j)$ for all $f\in\Cal F$. The inequality $$ P\(\sup_{f\in\Cal F}|S_n(f)|> A\sqrt n\sigma^2\)\le 2P\(\sup_{f\in\Cal F}|S_n(f)-\bar S_n(f)|> \frac23 A\sqrt n\sigma^2\). \tag7.3 $$ follows from Lemma~7.1 if we apply it for the countable sets $Z_n(f)=S_n(f)$ and $\bar Z_n(f)=\bar S_n(f)$, $f\in\Cal F$, of random variables and $x=\frac23 A\sqrt n\sigma^2$, $\alpha=\frac13 A\sqrt n\sigma^2$, since the fields $S_n(f)$ and $\bar S_n(f)$ are independent, and $P(|\bar S_n(f)|\le\alpha)>\frac12$ for all $f\in\Cal F$. Indeed, $\alpha=\frac13 A\sqrt n\sigma^2\ge \sqrt2\sigma$, $E\bar S_n(f)^2 \le\sigma^2$, thus Chebishev's inequality implies that $P(|\bar S_n(f)|\le\alpha)\ge P(|\bar S_n(f)|\le\sqrt2\sigma) \ge\frac12$ for all $f\in\Cal F$. Let us observe that the random field $$ S_n(f)-\bar S_n(f)=\frac1{\sqrt n}\sum_{j=1}^n \(f(\xi_j) -f(\bar\xi_j)\), \quad f\in \Cal F, \tag7.4 $$ and its randomization $$ \frac1{\sqrt n}\sum_{j=1}^n \e_j \(f(\xi_j) -f(\bar\xi_j)\), \quad f\in \Cal F, \tag$7.4'$ $$ have the same distribution. Indeed, even the conditional distribution of ($7.4'$) under the condition that the values of the $\e_j$-s are prescribed agrees with the distribution of (7.4) for all possible values of the $\e_j$-s. This follows from the observation that the distribution of the field (7.4) does not change if we exchange the random variables $\xi_j$ and $\bar\xi_j$ for certain indices $j$, and this corresponds to considering the conditional distribution of the field in ($7.4'$) under the condition that $\e_j=-1$ for these indices $j$, and $\e_j=1$ for the remaining ones. The above relation together with formula (7.3) imply that $$ \align &P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n f(\xi_j)\right| \ge A n^{1/2}\sigma^{2}\)\\ &\qquad \le 2P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n \e_j\[f(\xi_j)-\bar f(\xi_j)\]\right| \ge\frac23 A n^{1/2}\sigma^{2}\) \\ &\qquad\le 2P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n \e_jf(\xi_j)\right| \ge\frac A3 n^{1/2}\sigma^{2}\) \\ &\qquad\qquad+ 2P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left| \summ_{j=1}^n \e_jf(\bar\xi_j)\right| \ge\frac A3n^{1/2}\sigma^{2}\) \\ &\qquad=4P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n \e_jf(\xi_j)\right| \ge\frac A3n^{1/2}\sigma^{2}\). \endalign $$ Lemma~7.2 is proved. \medskip Let me briefly explain the approach to the proof of Proposition~6.2. We have to estimate a probability of the form $P\(n^{-1/2}\supp_{f\in\Cal F}\left|\summ_{j=1}^nf(\xi_j)\right|>u\)$, and by Lemma~7.2 this can be replaced by the estimation of the probability $P\(n^{-1/2}\supp_{f\in\Cal F}\left| \summ_{j=1}^n \e_jf(\xi_j)\right|>\frac u3\)$ with some independent random variables $\e_j$, $P(\e_j=1)=P(\e_j=-1)=\frac12$, $j=1,\dots,n$, which are also independent of the random variables $\xi_j$. We shall bound the conditional probability of the event appearing in this modified problem under the condition that the values of the random variables $\xi_j$ are prescribed. This can be done with the help of Hoeffding's inequality formulated in Theorem~3.4 and the $L_2$-density property of the class of functions $\Cal F$ we consider. By working out the details we are led to the estimation of the probability $P\(n^{-1/2}\supp_{f\in\Cal F'}\left| \summ_{j=1}^n f(\xi_j)\right|>u^{1+\alpha}\)$ with some new nice $L_2$-dense class of bounded functions $\Cal F'$ and some number $\alpha>0$. This problem is very similar to the original one, but it is simpler, since the number~$u$ is replaced by a larger number $u^{1+\alpha}$ in it. By repeating this argument successively, in finitely many steps we get the proof of Proposition~6.2. The above sketched argument suggests a backward induction procedure to prove Proposition~$6.2$. To carry out such a program first we introduce a property we want to prove. \medskip\noindent {\bf Definition of good tail behaviour for a class of normalized random sums.} {\it Let us fix some measurable space $(X,\Cal X)$ and a probability measure $\mu$ on it together with some integer $n\ge2$ and real number $\sigma>0$, and consider some class $\Cal F$ of functions $f(x)$ on the space $(X,\Cal X)$. Take a sequence of independent $\mu$ distributed random variables $\xi_1,\dots,\xi_n$, and define with its help the normalized random sums $S_n(f)=\frac1{\sqrt n} \summ_{j=1}^nf(\xi_j)$, $f\in \Cal F$. Given some real number $T>0$ we say that the set of normalized random sums $S_n(f)$ determined by the class of functions $\Cal F$ has a good tail behaviour at level~$T$ (with parameters $n$ and $\sigma^2$ which we shall fix in the sequel) if the inequality $$ P\(\sup_{f\in\Cal F}|S_n(f)|\ge A \sqrt n\sigma^2\) \le \exp\left\{-A^{1/2}n\sigma^2 \right\} \tag7.5 $$ holds for all numbers $A>T$.} \medskip Now we formulate Proposition 7.3 and show that Proposition 6.2 follows from it. \medskip\noindent {\bf Proposition 7.3.} {\it Let us fix a positive integer~$n\ge2$, a real number $\sigma>0$ and a probability measure $\mu$ on a measurable space $(X,\Cal X)$ together with a countable $L_2$-dense class $\Cal F$ of functions $f=f(x)$ on the space $(X,\Cal X)$ with some prescribed exponent $L\ge1$ and parameter~$D$. Let us also assume that all functions $f\in \Cal F$ satisfy the conditions $\supp_{x\in X}|f(x)|\le\frac14$, $\int f^2(x)\mu(\,dx)\le\sigma^2$, and $n\sigma^2>K(L+\beta)\log n$ with a sufficiently large fixed number~$K$ and $\beta=\max\(\frac{\log D}{\log n},0\)$. If there is a number $T>1$ such that for all classes of functions $\Cal F$ which satisfy the above conditions the class of normalized random sums $S_n(f)=\frac1{\sqrt n}\summ_{j=1}^n f(\xi_j)$, $f\in\Cal F$, defined with the help of a sequence of independent $\mu$ distributed random variables $\xi_1,\dots,\xi_n$ have a good tail behaviour at level~$T$, then there is a universal constant~$\bar A_0$ such that the number ~$\bar T=T^{3/4}$ also have this property provided that $T\ge\bar A_0$. We can choose for instance $\bar A_0=64\cdot 10^{12}$ and $K=1$.} \medskip Proposition~6.2 simply follows from Proposition~7.3. To show this let us first observe that the class of normalized random sums $S_n(f)$, $f\in\Cal F$, has a good tail behaviour at level $T_0=\frac1{4\sigma^2}$ if the class of functions $\Cal F$ satisfies the conditions of Proposition~7.3. Indeed, in this case $P\(\supp_{f\in\Cal F}|S_n(f)| \ge A\sqrt n\sigma^2\)\le P\(\supp_{f\in\Cal F}|S_n(f)|> \frac{\sqrt n}4\)=0$ for all $A>T_0$. Then the repetitive application of Proposition~7.3 yields that the class of random sums $S_n(f)$ has a good tail behaviour at all levels $T\ge T_0^{(3/4)^j}$ if $T_0^{(3/4)^j}\ge\bar A_0$, hence for $T=\bar A_0^{4/3}$ if the class of functions $\Cal F$ satisfies the conditions of Proposition~7.3. If the class of functions $f\in\Cal F$ satisfies the conditions of Proposition~6.2, then the class of functions $\bar{\Cal F}=\left\{\bar f=\frac f4\: f\in\Cal F\right\}$ satisfies the conditions of Proposition~7.2, (actually with $\bar\sigma=\frac\sigma4$, and a better parameter~$D$ for the class $\Cal F$), hence the class of functions $S_n(\bar f)$, $\bar f\in \bar{\Cal F}$, has a good tail behaviour at level $T=\bar A_0^{4/3}$. This implies that the original class of functions $\Cal F$ satisfy formula (6.3) in Proposition~6.2 with $4K$, $A_0=4\bar A_0^{4/3}$ and $\gamma=\frac12$, and this is what we had to show. \medskip\noindent {\it The proof of Proposition 7.3.}\/ Fix a class of functions $\Cal F$ which satisfies the conditions of Proposition~7.3 together with two independent sequences $\xi_1,\dots,\xi_n$ and $\e_1,\dots,\e_n$ of independent random variables, where $\xi_j$ is $\mu$-distributed, $P(\e_j=1)=P(\e_j=-1)=\frac12$, $1\le j\le n$, and investigate the conditional probability $$ P(f,A|\xi_1,\dots,\xi_n)=P\(\left.\frac1{\sqrt n}\left|\summ_{j=1}^n \e_jf(\xi_j)\right| \ge \frac A6\sqrt n\sigma^2\right|\xi_1,\dots,\xi_n\) $$ for all functions $f\in\Cal F$, $A\ge T$ and values $(\xi_1,\dots,\xi_n)$ in the condition. By the Hoeffding inequality presented in Theorem~3.4 $$ P(f,A|\xi_1,\dots,\xi_n)\le 2\exp\left\{-\frac{\frac 1{36} A^2 n\sigma^4}{2\bar S^2(f,\xi_1,\dots,\xi_n)}\right\} \tag7.6 $$ with $$ \bar S^2(f,x_1,\dots,x_n)=\frac1n\sum_{j=1}^n f^2(x_j), \quad f\in \Cal F. $$ Let us introduce the set $$ H=H(A)=\left\{(x_1,\dots,x_n)\: \sup_{f\in\Cal F} \bar S^2(f,x_1,\dots,x_n)\ge \(1+A^{4/3}\)\sigma^2\right\}. \tag7.7 $$ I claim that $$ P((\xi_1,\dots,\xi_n)\in H)\le e^{-A^{2/3} n\sigma^2}\quad\text{ if } A\ge T. \tag$7.7'$ $$ (The set $H$ plays the role of the small exceptional set, where we cannot provide a good estimate for $P(f,A|\xi_1,\dots,\xi_n)$ for some $f\in\Cal F$.) To prove relation ($7.7'$) let us consider the functions $\bar f=\bar f(f)$, $\bar f(x)=f^2(x)-\int f^2(x)\mu(\,dx)$, and introduce the class of functions $\Cal F'=\{\bar f(f)\: f\in\Cal F\}$. Let us show that the class of functions $\Cal F'$ satisfies the conditions of Proposition~7.3, hence the estimate (7.5) holds for the class of functions $\Cal F'$ if $A\ge T^{4/3}$. The relation $\int \bar f(x)\mu(\,dx)=0$ clearly holds. The condition $\sup| \bar f(x)|\le\frac 18<\frac14$ also holds if $\sup |f(x)|\le \frac14$, and $\int \bar f^2(x)\mu(\,dx)\le \int f^4(x)\mu(\,dx)\le \frac 1{16}\int f^2(x)\,\mu(\,dx)\le\frac{\sigma^2}{16}<\sigma^2$ if $f\in\Cal F$. It remained to show that $\Cal F'$ is an $L_2$-dense class with exponent $L$ and parameter $D$. To show this observe that $\int (\bar f(x)-\bar g(x))^2\rho(\,dx)\le 2\int(f^2(x)-g^2(x))^2\rho(\,dx)+ 2\int(f^2(x)-g^2(x))^2\mu(\,dx)\le2 (\supp (|f(x)|+|g(x)|)^2 \(\int (f(x)-g(x))^2(\rho(\,dx)+\mu(\,dx)\)\le \int (f(x)-g(x))^2\bar\rho(\,dx)$ for all $f, g\in\Cal F$, $\bar f=\bar f(f)$, $\bar g=\bar g(g)$ and probability measure $\rho$, where $\bar\rho=\frac{\rho+\mu}2$. This means that if $\{f_1,\dots,f_m\}$ is an $\e$-dense subset of $\Cal F$ in the space $L_2(X,\Cal X,\bar\rho)$, then $\{\bar f_1,\dots,\bar f_m\}$ is an $\e$-dense subset of $\Cal F'$ in the space $L_2(X,\Cal X,\rho)$, and not only $\Cal F$, but also $\Cal F'$ is an $L_2$-dense class with exponent $L$ and parameter $D$. We get by applying the inductive hypothesis of Proposition 7.3 for the number $A^{4/3}\ge T^{4/3}$ and the class of functions $\Cal F'$ that $$ \align P((\xi_1,\dots,\xi_n)\in H)&=P\(\sup_{f\in\Cal F} \(\frac1n \sum_{j=1}^n \bar f(\xi_j) +\frac1n \sum_{j=1}^n E f^2(\xi_j)\) \ge \(1+A^{4/3}\)\sigma^2\)\\ &\le P\(\sup_{\bar f\in\bar {\Cal F}} \frac1{\sqrt n} \sum_{j=1}^n \bar f(\xi_j) \ge A^{4/3}n^{1/2}\sigma^2\) \le e^{-A^{2/3} n\sigma^2}, \endalign $$ i.e. relation ($7.7'$) holds. Formula (7.6) and the definition of the set $H$ given in (7.7) yield the estimate $$ P(f,A|\xi_1,\dots,\xi_n)\le 2e^{- A^{2/3} n\sigma^2/144} \quad \text{if }(\xi_1,\dots,\xi_n)\notin H \tag7.8 $$ for all $f\in \Cal F$ and $A\ge T\ge1$. (Here we used the estimate $1+A^{4/3}\le2A^{4/3}$.) Let us introduce the conditional probability $$ P(\Cal F,A|\xi_1,\dots,\xi_n)= P\(\left.\sup_{f\in \Cal F} \frac1{\sqrt n}\left|\summ_{j=1}^n \e_jf(\xi_j)\right| \ge \frac A3\sqrt n\sigma^2\right|\xi_1,\dots,\xi_n\) $$ for all $(\xi_1,\dots,\xi_n)$ and $A\ge T$. We shall estimate this conditional probability with the help of relation (7.8) if $(\xi_1,\dots,\xi_n) \notin H$. Given some set of $n$~points $(x_1,\dots,x_n)$ in the space $(X,\Cal X)$ let us introduce the measure $\nu=\nu(x_1,\dots,x_n)$ on $(X,\Cal X)$ in such a way that $\nu$ is concentrated in the points $x_1,\dots,x_n$, and $\nu(\{x_j\})=\frac1n$. If $\int f^2(x)\nu(\,dx)\le\delta^2$ for a function $f$, then $\left|\frac1{\sqrt n}\summ_{j=1}^n \e_jf(x_j)\right|\le n^{1/2}\int|f(x)|\nu(\,dx)\le n^{1/2}\delta$. Since the condition $n\sigma^2\ge K(L+\beta)\log n$ in Proposition 7.3 also implies that $n\sigma^2\ge1$ (if the constant $K$ is chosen sufficiently large), the above estimate implies that if $f$ and $g$ are two functions such that $\int (f-g)^2\nu(\,dx)\le \delta^2$ with $\delta=\frac A{6n}$, then $\left|\frac1{\sqrt n}\summ_{j=1}^n \e_jf(x_j)-\frac1{\sqrt n}\summ_{j=1}^n \e_jg(x_j)\right| \le\frac A{6\sqrt n}\le\frac A6 \sqrt n\sigma^2$. Let us fix some (random) point $(\xi_1,\dots,\xi_n)\notin H$, consider the measure $\nu=\nu(\xi_1,\dots,\xi_n)$ corresponding to it and choose a $\bar\delta$-dense subset $\{f_1,\dots,f_m\}$ of $\Cal F$ in the space $L_2(X,\Cal X,\nu)$ with $\bar\delta=\frac1{6n}\le\delta= \frac A{6n}$, whose cardinality $m$ satisfies the inequality $m\le D\bar\delta^{-L}$. This is possible because of the $L_2$-dense property of the class~$\Cal F$. (This is the point where the $L_2$-dense property of the class of functions $\Cal F$ is exploited in its full strength.) The above facts imply that if $\frac1{\sqrt n}\left|\summ_{j=1}^n \e_jf(\xi_j)\right| \ge \frac A3\sqrt n\sigma^2$ for some function $f\in \Cal F$, then $\frac1{\sqrt n}\left|\summ_{j=1}^n \e_jf_l(\xi_j)\right|\ge\frac A6\sqrt n\sigma^2$ for some function $f_l$ of the $\bar\delta$-dense subset $\{f_1,\dots,f_m\}$ of $\Cal F$ with the fixed point $(\xi_1,\dots,\xi_n)\notin H$. Hence $P(\Cal F,A|\xi_1,\dots,\xi_n) \le\summ_{l=1}^m P(f_l,A|\xi_1,\dots,\xi_n)$ with these functions $f_1,\dots,f_m$, and relation (7.8) yields that $$ P(\Cal F,A|\xi_1,\dots,\xi_n)\le 2D(6n)^Le^{- A^{2/3} n\sigma^2/144} \quad \text{if }(\xi_1,\dots,\xi_n)\notin H \text{ and } A\ge T. $$ This inequality together with Lemma~7.2 (under the restriction that $A\ge\bar A_0\ge\frac{3\sqrt 2}{\sqrt n\sigma}\ge3\sqrt2$) and estimate~($7.7'$) imply that $$ \aligned &P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n f(\xi_j)\right| \ge A n^{1/2}\sigma^{2}\) \le 4P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n \e_jf(\xi_j)\right| \ge \frac A3 n^{1/2}\sigma^{2}\)\\ &\qquad \le 8D(6n)^Le^{-A^{2/3}n\sigma^2/144} +4e^{-A^{2/3}n\sigma^2} \quad \text{if } A\ge T. \endaligned \tag7.9 $$ By the condition $n\sigma^2\ge K(L+\beta)\log n=KL\log n+K\log(\max(D,1))$, hence the first term at the right-hand side of (7.9) can be bounded as $$ \align &8D(6n)^Le^{-A^{2/3}n\sigma^2/144}\\ &\qquad \le e^{-A^{1/2}n\sigma^2}\cdot8D6^L n^{L(1-A^{1/2}/3)}\max(D,1)^{-A^{1/2}/3} \le \tfrac12e^{-A^{1/2}n\sigma^2} \endalign $$ if $A\ge T\ge \bar A_0\ge64\cdot10^{12}$ and $K\ge1$. (With such parameters $\frac{A^{2/3}}{144}-A^{1/2}\le\frac13A^{1/2}$.) With such a choice of the parameters the inequality $\frac{3\sqrt2} {\sqrt n\sigma}\le \frac{3\sqrt2}{\sqrt{K\log2}}\le \bar A_0\le A$, needed for the validity of relation (7.2), also holds. The second term at the right-hand side of (7.9) be bounded as $4e^{-A^{2/3}n\sigma^2}\le \frac12e^{-A^{1/2}n\sigma^2}$. with the above choice of the numbers $\bar A_0$ and $K$. By the above calculation formula (7.9) yields the inequality $$ P\(\frac1{\sqrt n}\supp_{f\in\Cal F}\left|\summ_{j=1}^n f(\xi_j)\right| \ge An^{1/2}\sigma^{2}\) \le e^{-A^{1/2}n\sigma^2} $$ if $A\ge T$, and the constants $\bar A_0$ and $K$ are chosen sufficiently large, for instance $\bar A_0=64\cdot 10^{12}$ and $K=1$ is an appropriate choice. \beginsection 8. Formulation of the main results of this work This section contains the main results of this work about multiple stochastic integrals and their supremum. Section~4 contains these results in the special case of one-fold integrals together with their version about the supremum of appropriate classes of normalized sums of independent and identically distributed random variables with zero expectation. (See Theorem~4.1 and~$4.1'$ and their comparison.) The results about multiple stochastic integrals also have a similar version, and they will be also presented. Here the role of sums of independent, and identically distributed random variables are taken by degenerate $U$-statistics of independent and identically distributed random variables. The condition that the $U$-statistics have to be degenerate plays the a role similar to the condition about the zero expectation of the summands when the independent sum versions of the one-fold integral results are considered. The basic notions about $U$-statistics needed to understand the results will also be explained. The proof of the equivalence of the results about multiple integrals and $U$-statistics formulated in this section requires a detailed study of the property of $U$-statistics, a problem which has a special interest in itself. This will be the subject of the next section. We also formulate some results about multiple Wiener--It\^o integrals which are natural analogs of the results about multiple integrals with respect to normalized empirical measures. But these results are only briefly discussed, because they do not belong to the main subject of this work, and they demand a more detailed study of multiple Wiener--It\^o integrals. Finally, this section is finished with a the two-dimensional version of Example~3.2 which shows that certain conditions of the results discussed here are really necessary. \medskip Let us consider a sequence of iid. random variables $\xi_1,\dots,\xi_n$ taking values on a measurable space $(X,\Cal X)$. Let $\mu$ denote its distribution, and introduce the empirical distribution of this sequence defined in (4.5). Given a measurable function $f(x_1,\dots,x_k)$ on the $k$-fold product space $(X^k,\Cal X^k)$ introduce its integral $J_{n,k}(f)$ with respect to the $k$-fold product of the normalized empirical measure $\sqrt n(\mu_n-\mu)$ defined in formula (4.8). Here we define the domain of integration by deleting the diagonals $x_j=x_l$, $1\le j0$ and $\alpha=\alpha_k>0$, such that the random integral $J_{n,k}(f)$ defined by formulas (4.5) and (4.8) satisfies the inequality $$ P\(|J_{n,k}(f)|>u\)\le C \max\(\exp\left\{-\alpha \(\frac u\sigma\)^{2/k}\right\}, \exp\left\{-\alpha (nu^2)^{1/(k+1)}\right\} \) \tag8.3 $$ for all $u>0$. The constants $C=C_k>0$ and $\alpha=\alpha_k>0$ in formula (8.3) depend only on the parameter~$k$.}\medskip Theorem 8.1 can be reformulated in the following equivalent form. \medskip\noindent {\bf Theorem 8.1$'$.} {\it Under the conditions of Theorem 8.1 $$ P\(|J_{n,k}(f)|>u\)\le C \exp\left\{-\alpha \(\frac u\sigma\)^{2/k}\right\} \quad \text{for all } 00$, $\alpha=\alpha_k>0$, depending only on the multiplicity~$k$ of the integral $J_{n,k}(f)$.} \medskip Theorem 8.1 clearly implies Theorem~$8.1'$, since in the case $u\le n^{k/2}\sigma^{k+1}$ the first term is larger than the second one in the maximum at the right-hand side of formula~(8.3). On the other hand Theorem~$8.1'$ implies Theorem~8.1 also if $u>n^{k/2}\sigma^{k+1}$, since in this case Theorem~$8.1'$ can be applied with $\bar\sigma=\(u n^{-k/2}\)^{1/(k+1)}\ge \sigma$. This yields that $P\(|J_{n,k}(f)|>u\)\le C \exp\left\{-\alpha \(\frac u{\bar\sigma}\)^{2/k}\right\}=C \exp\left\{-\alpha (nu^2)^{1/(k+1)}\right\}$ if $u>n^{k/2}\sigma^{k+1}$. Theorem~8.1 or Theorem~$8.1'$ state that the tail probability $P(|J_{n,k}(f)|>u)$ of the $k$-fold random integral $J_{n,k}(f)$ can be bounded similarly to the probability $P(|\const\sigma\eta^k|>u)$, where $\eta$ is a random variable with standard normal distribution and $\sigma$ is the number appearing in relation (8.2), provided that the level~$u$ we consider is less than $n^{k/2}\sigma^{k+1}$. (The value of the number $\sigma^2$ in formula (8.2) is closely related to the variance of $J_{n,k}(f)$.) At the end of this section an example is given which shows that such a condition is really needed in the above results. Now we formulate Theorem 8.2 which is the generalization of Theorem~4.1 for multiple integrals. Here we apply the notions of $L_2$-dense classes and countably approximability introduced in Section~4. \medskip\noindent {\bf Theorem 8.2.} {\it Let us have a non-atomic probability measure $\mu$ on a measurable space $(X,\Cal X)$ together with a countable and $L_2$-dense class $\Cal F$ of functions $f=f(x_1,\dots,x_k)$ of $k$ variables with some parameter $D$ and exponent $L$, $L\ge1$, on the product space $(X^k,\Cal X^k)$ which satisfies the conditions $$ \|f\|_\infty=\supp_{x_j\in X,\;1\le j\le k}|f(x_1,\dots,x_k)|\le 1, \qquad \text{for all } f\in \Cal F \tag8.4 $$ and $$ \|f\|_2^2=Ef^2(\xi_1,\dots,\xi_k)=\int f^2(x_1,\dots,x_k) \mu(\,dx_1)\dots\mu(\,dx_k)\le \sigma^2 \qquad \text{for all } f\in \Cal F \tag8.5 $$ with some constant $0<\sigma\le1$. Then there exist some constants $C=C(k)>0$, $\alpha=\alpha(k)>0$ and $M=M(k)>0$ depending only on the parameter $k$ such that the supremum of the random integrals $J_{n,k}(f)$, $f\in \Cal F$, defined by formula (4.8) satisfies the inequality $$ \aligned P\(\supp_{f\in\Cal F}|J_{n,k}(f)|\ge u\)&\le CD \exp\left\{-\alpha \(\frac u{\sigma}\)^{2/k}\right\} \\ &\qquad \text{if}\quad n\sigma^2\ge \(\frac u\sigma\)^{2/k} \ge M(L+\beta)^{3/2}\log\frac2\sigma, \endaligned \tag8.6 $$ where $\beta=\max\(\frac{\log D}{\log n},0\)$ and the numbers $D$ and $L$ agree with the parameter and exponent of the $L_2$-dense class~$\Cal F$. The condition about the countable cardinality of the class $\Cal F$ can be replaced by the weaker condition that the class of random variables $J_{n,k}(f)$, $f\in\Cal F$, is countably approximable.} \medskip To formulate that version of Theorems~8.1 and~8.2 which corresponds to the results about sums of independent random variables in the case $k=1$ let us introduce the following notions: \medskip\noindent {\bf Definition of $U$-statistics.} {\it Let us consider a function $f=f(x_1,\dots,x_k)$ on the $k$-th power $(X^k,\Cal X^k)$ of a space $(X,\Cal X)$ together with a sequence of independent and identically distributed random variables $\xi_1,\dots,\xi_n$, $n\ge k$, which take their values on this space $(X,\Cal X)$. The expression $$ I_{n,k}(f)=\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb f\(\xi_{l_1},\dots,\xi_{l_k}\) \tag8.7 $$ is called a $U$-statistic of order $k$ with the sequence $\xi_1,\dots,\xi_n$, and $f$ is called its kernel function. To make our later notation non-ambiguous let us also consider functions of the form $f(x_{u_1},\dots,x_{u_k})$, that is let us allow the possibility that the variables of the function~$f$ which take their values in the space $(X,\Cal X)$ are indexed in a general way. In the case of such an indexation we define $$ I_{n,k}(f)=\frac1{k!}\summ\Sb 1\le l_{u_j}\le n,\; j=1,\dots,k\\ l_{u_j}\neq l_{u'_j} \text{ if } j\neq j'\endSb f\(\xi_{l_{u_1}},\dots,\xi_{l_{u_k}}\). \tag$8.7'$ $$ A similar convention will be applied in the definition of decoupled $U$-statistics introduced later, and the following definition of degenerate $U$-statistics and canonical functions can also be similarly reformulated in the case of general indexation.} \medskip The degenerate $U$-statistics which correspond to sums of identically distributed random variables with expectation zero constitute an important subclass of the $U$-statistics. We define it together with the notion of canonical kernel function which is closely related to it. \medskip\noindent {\bf Definition of degenerate $U$-statistics.} {\it A $U$-statistic $I_{n,k}(f)$ of order~$k$ with a sequence of iid. random variables $\xi_1,\dots,\xi_n$ is called degenerate if its kernel function $f(x_1,\dots,x_k)$ satisfies the relation $$ \align &Ef(\xi_1,\dots,\xi_k|\xi_1=x_1,\dots,\xi_{j-1}=x_{j-1}, \xi_{j+1}=x_{j+1},\dots,\xi_k=x_k)=0 \\ &\qquad\qquad \text{for all } 1\le j\le k \text { and } x_s\in X, \; s\neq j. \endalign $$ }\medskip \noindent {\bf Definition of canonical kernel function.} {\it A function $f(x_1,\dots,x_k)$ taking values on the $k$-fold product of a measure space $(X,\Cal X)$ is called a canonical function with respect to a probability measure $\mu$ on $(X,\Cal X)$ if $$ \int f(x_1,\dots,x_{j-1},u,x_{j+1},\dots,x_k)\mu(\,du)=0\quad \text{for all } 1\le j\le k \text{ \ and \ } x_s\in X, \; s\neq j. \tag8.8 $$ }\medskip It is clear that a $U$-statistic $I_{n,k}(f)$ with kernel function $f$ and independent $\mu$-distributed random variables $\xi_1,\dots,\xi_n$ is degenerate if and only if its kernel function is canonical with respect to the probability measure~$\mu$. Now we can formulate two results about $U$-statistics which are, as we shall see in the next section, equivalent to Theorems~8.1 and~8.2. \medskip\noindent {\bf Theorem 8.3.} {\it Let us have a measurable function $f(x_1,\dots,x_k)$ on the $k$-fold product $(X^k,\Cal X^k)$, $k\ge1$, of a measure space $(X,\Cal X)$ with some $k\ge1$ together with a probability measure $\mu$ on $(X,\Cal X)$ and a sequence of iid.\ random variables $\xi_1,\dots,\xi_n$ with distribution~$\mu$ on $(X,\Cal X)$. Let us consider the $k$-fold $U$-statistic $I_{k,n}(f)$ with this sequence of random variables $\xi_1,\dots,\xi_n$. Assume that this $U$-statistic is degenerate, i.e. the kernel function $f(x_1,\dots,x_k)$ of this $U$-statistic is canonical with respect to the measure $\mu$. Let us also assume that the function $f$ satisfies conditions (8.1) and (8.2) with some number $0<\sigma\le1$. Then there exist some constants $C=C_k>0$ and $\alpha=\alpha_k>0$ such that the inequality $$ P\(n^{-k/2}|I_{n,k}(f)|>u\)\le C \exp\left\{-\alpha \(\frac u\sigma\)^{2/k}\right\} \tag8.9 $$ holds for all $00$ and $\alpha=\alpha_k>0$ depend only on the parameter~$k$.} \medskip\noindent {\bf Theorem 8.4.} {\it Let us have a probability measure $\mu$ on a measurable space $(X,\Cal X)$ together with a countable and $L_2$-dense class $\Cal F$ of functions $f=f(x_1,\dots,x_k)$ of $k$ variables with some parameter $D$ and exponent $L$, $L\ge1$, on the product space $(X^k,\Cal X^k)$ which satisfies conditions (8.4) and (8.5) with some constant $0<\sigma\le1$. Beside these conditions let us assume that for a sequence of independent $\mu$ distributed random variables $\xi_1,\dots,\xi_n$ the $U$-statistics $I_{n,k}(f)$ with this sequence $\xi_1,\dots,\xi_n$ are degenerate for all $f\in\Cal F$, or in an equivalent form all functions $f\in \Cal F$ are canonical with respect to the measure~$\mu$. Then there exist some constants $C=C(k)>0$, $\alpha=\alpha(k)>0$ and $M=M(k)>0$ depending only on the parameter $k$ such that the inequality $$ \aligned P\(\supp_{f\in\Cal F}n^{-k/2}|I_{n,k}(f)|\ge u\)&\le CD \exp\left\{-\alpha \(\frac u{\sigma}\)^{2/k}\right\} \\ &\qquad \text{if}\quad n\sigma^2\ge \(\frac u\sigma\)^{2/k} \ge M(L+\beta)^{3/2}\log\frac2\sigma, \endaligned \tag8.10 $$ holds, where $\beta=\max\(\frac{\log D}{\log n},0\)$ and the number $D$ and $L$ agree with the parameter and exponent of the $L_2$-dense class~$\Cal F$. The condition about the countable cardinality of the class $\Cal F$ can be replaced by the weaker condition that the class of random variables $n^{-k/2}I_{n,k}(f)$, $f\in\Cal F$, is countably approximable.} \medskip Let us briefly describe the Gaussian counterpart of the above results. Here some basic notions and results about multiple Wiener--It\^o integrals are applied. But since the results about these Gaussian fields do not belong to the main subject of this work, they are mainly interesting for us for the sake of a comparison, most technical details will be omitted from our discussion. Let us consider a measurable space $(X,\Cal X)$ together with a non-atomic $\sigma$-finite measure $\mu$ on it. Let $Z_\mu$ be an orthogonal Gaussian random measure with counting measure $\mu$ on $(X,\Cal X)$, i.e.\ assume that the random variables $Z_\mu(A)$, $A\in\Cal X$, $\mu(A)<\infty$ are defined, they are jointly Gaussian, $EZ_\mu(A)=0$ for all $A\in\Cal A$, $\mu(A)<\infty$ and $EZ_\mu(A)Z_\mu(B)=\mu(A\cap B)$ for all $A\in\Cal A$, $B\in\Cal A$, $\mu(A)<\infty$, $\mu(B)<\infty$. Let us observe that these relations imply that if $A\in\Cal X$, $\mu(A)<\infty$ and $B\in \Cal X$, $\mu(B)<\infty$ are disjoint sets, then $Z_\mu(A)$ and $Z_\mu(B)$ are independent, and $Z_\mu(A\cup B)=Z_\mu(A)+Z_\mu(B)$ with probability~1. The last relation follows from the fact that $E(Z_\mu(A\cup B)-Z_\mu(A)-Z_\mu(B))^2=0$ under these conditions. If $f(x_1,\dots,x_k)$ is a measurable function on $(X^k,\Cal X^k)$ such that $$ \int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)<\infty, $$ then the multiple Wiener--It\^o integral $Z_{\mu,k}(f)=\frac1{k!}\int f(x_1,\dots,x_k)Z_\mu(\,dx_1)\dots Z_\mu(\,dx_k)$ can be defined, and it satisfies similar estimates as the random integrals $J_{n,k}(f)$. This statement will be formulated more explicitly in the following Theorem~8.5. \medskip\noindent {\bf Theorem 8.5} {\it Let us fix a measurable space $(X,\Cal X)$ together with a $\sigma$-finite non-atomic measure~$\mu$ on it, and let $Z_\mu$ be an orthogonal Gaussian random measure with counting measure $\mu$ on $(X,\Cal X)$. If $f(x_1,\dots,x_k)$ is a measurable function on $(X^k,\Cal X^k)$ such that $\frac1{k!}\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)\le\sigma^2$ with some $0<\sigma<\infty$, then $$ P(|Z_{\mu,k}(f)|>u)\le C \exp\left\{-\alpha\(\frac u\sigma\)^{2/k}\right\} \tag8.11 $$ for all $u>0$ with some constants $C=C(k)$ and $\alpha=\alpha(k)$ depending only on~$k$. If $\Cal F$ is a countable class of functions of $k$ variables on $(X,\Cal X)$ such that $$ \int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots \mu(\,dx_k)\le\sigma^2 \quad \text{\rm with some } 0<\sigma\le1 \text { \rm for all } f\in \Cal F, $$ and there exist some constant $D>0$ and $L>0$ such that a subset $\{f_1,\dots,f_m\}\subset\Cal F$ can be chosen with $m\le D\e^{-L}$ elements for which $$ \min_{1\le j\le m}\int\(f(x_1,\dots,x_k)-f_j(x_1,\dots,x_k)\)^2 \mu(\,dx_1)\dots\mu(\,dx_k)\le\e \quad \text{ \rm for all } f\in\Cal F, $$ then the inequality $$ P\(\sup_{f\in \Cal F}|Z_{\mu,k}(f)|>u\)\le C(D+1) \exp\left\{-\alpha\(\frac u\sigma\)^{2/k}\right\} \quad \text{if }u\ge ML^{k/2}\sigma \log^{k/2}\frac2\sigma \tag8.12 $$ holds with some universal constants $C=C(k)>0$, $M=M(k)>0$ and $\alpha=\alpha(k)>0$.} \medskip Since the above result does not belong to the main part of this work, only a sketchy proof will be presented. Nelson's inequality, mentioned at the start of this section will be formulated and proved in the Appendix, and it will be explained how Theorem~8.5 can be proved with its help. The above results show that multiple integrals with respect to a normalized empirical measure or degenerate $U$-statistics satisfy some estimates similar to multiple Wiener--It\^o integrals, but they hold under more restrictive conditions. This difference between multiple integrals with respect to a normalized empirical measure and orthogonal Gaussian measures can be explained similarly to some arguments presented in Section~4 about the one-fold integral case. Here we do not repeat them, we only give an example similar to Example~3.2 which shows that the condition $u\le n^{k/2} \sigma^{k+1}$ cannot be dropped from the conditions of Theorem~8.2. For the sake of simplicity we restrict our attention to the case $k=2$. \medskip\noindent {\bf Example 8.6.} {\it Let $\xi_1,\dots,\xi_n$ be a sequence of independent, identically distributed random variables taking values on the plane $R^2=X$ such that $\xi_j=(\eta_{j,1},\eta_{j,2})$, $\eta_{j,1}$ and $\eta_{j,2}$ are independent, $P(\eta_{j,1}=1)=P(\eta_{j,1}=-1) =\frac{\sigma^2}8$, $P(\eta_{j,1}=0)=1-\frac{\sigma^2}4$, $P(\eta_{j,2}=1)=P(\eta_{j,2}=-1)=\frac12$ for all $1\le j\le n$, introduce the function $f(x,y)=f((x_1,x_2),(y_1,y_2))=x_1y_2+x_2y_1$, $x=(x_1,x_2)\in R^2$, $y=(y_1,y_2)\in R^2$, and define the $U$-statistic $$ I_{n,2}(f)=\sum_{1\le j,k\le n,\,j\neq k} (\eta_{j,1}\eta_{k,2}+\eta_{k,1}\eta_{j,2}) $$ of order 2 with the above kernel function $f$ and the sequence of independent random variables $\xi_1,\dots,\xi_n$. Then $I_{n,2}(f)$ is a degenerate $U$-statistic. If $u\ge B_1n\sigma^3$ with some appropriate constant $B_1>0$, $\bar B_2^{-1}n\ge u\ge \bar B_2 n^{-2}$ with a sufficiently large fixed number $\bar B_2>0$ and $\sigma\ge\frac1n$, then the estimate $$ P(n^{-1}I_{n,2}(f)>u)\ge \exp\left\{-Bn^{1/3}u^{2/3}\log \(\frac u{n\sigma^3}\)\right\} \tag8.13 $$ holds with some $B>0$.}\medskip\noindent {\it Remark:}\/ The main content of the above example is that in the case $k=2$ the condition $\frac u\sigma\le n\sigma^2$ cannot be dropped from Theorem 8.3. Let us observe that in the case $u=n\sigma^3$ the right-hand side of (8.13) has the same order as Theorem 8.3 suggests. (In this model $\int f^2(x,y)\mu(\,dx)\mu(\,dy) =E(2\eta_{j,1}\eta_{j,2})^2=\sigma^2$.) If we consider the probability in (8.13) at the same level $u$, but with a much smaller parameter $\sigma^2$, then the probability at the right-hand side of (8.13) has a relatively small decrease, and the estimate of Theorem~8.3 does not hold any longer. Let me also remark that under some mild additional restrictions the estimate (8.13) can be slightly improved, the term $\log$ can be replaced by $\log^{2/3}$ in the exponent of the right-hand side of (8.13). To get this improvement some more calculation is needed, and the numbers $u_1$ and $u_2$ in the following calculations have to be replaced by $v_1=8n^{1/3}u^{2/3}\log^{-1/3}\(\frac u{n\sigma^3}\)$ and $v_2=\frac14n^{2/3}u^{1/3}\log^{1/3}\(\frac u{n\sigma^3}\)$. \medskip It is simple to check that the $U$-statistic we considered in the above example is degenerate because of the independence properties of the model and the relation $E\eta_{j,1}=E\eta_{j,2}=0$. In the proof of the estimate (8.13) we shall apply for one hand the results of Section~3, in particular Example~3.2 for the sequence $\eta_{j,1}$, $j=1,2,\dots,n$, on the other hand the following result from the theory of large deviations: If $X_1,\dots,X_n$ are iid. random variables, $P(X_1=1)=P(X_1=-1)=\frac12$, then for any number $0\le \alpha<1$ there exists some numbers $C_1=C_1(\alpha)>0$ and $C_2=C_2(\alpha)>0$ such that $P\(\summ_{j=1}^nX_j >u\)\ge C_1e^{-C_2u^2/n}$ for all $0\le u\le \alpha n$. \medskip\noindent {\it Proof of the statement of the example.}\/ We can write $$ P(n^{-1}I_{n,2}(f)>u)\ge P\(2\(\sum_{j=1}^n\eta_{j,1}\) \(\sum_{j=1}^n\eta_{j,2}\)>2nu\) -P\(2\sum_{j=1}^n\eta_{j,1}\eta_{j,2}>nu\). \tag8.14 $$ Because of the independence of the random variables $\eta_{j,1}$ and $\eta_{j,2}$ the first probability at the right-hand side of (8.14) can be bounded from below with the choice $v_1=8n^{1/3}u^{2/3}$ and $v_2=\frac18n^{2/3}u^{1/3}$ by means of Example 3.2. (The estimate of Example 3.2 can be applied with the choice $y=v_1$, since by the inequality $\frac n8\ge v_1\ge n\sigma^2$ the conditions of Example~3.2 are satisfied), together with the large-deviation result mentioned after the remark. These estimates together yield that $$ \align &P\(\(\sum_{j=1}^n\eta_{j,1}\)\(\sum_{j=1}^n\eta_{j,2}\)>2nu\)\ge P\(\sum_{j=1}^n\eta_{j,1} >v_1\)P\(\sum_{j=1}^n\eta_{j,2}>v_2\) \\ &\qquad \ge \exp\left\{-B_1v_1\log \(\frac{v_1}{n\sigma^2}\)-B_2\frac{v_2^2}{n}\right\} \ge\exp\left\{-B_3n^{1/3}u^{2/3}\log\(\frac u{n\sigma^3}\)\right\} \endalign $$ with appropriate constants $B_1>0$, $B_2>0$ and $B_3>0$. On the other hand by applying Bennett's inequality, more precisely its consequence given in formula (3.4) for the sum of the random variables $X_j=2\eta_{j,1}\eta_{j,2}$ and $y=nu$ we get the following upper bound for the second term at the right-hand side of (8.14): $$ \align P\(2\sum_{j=1}^n\eta_{j,1}\eta_{j,2}>nu\)&\le \exp\left\{-B_4nu\log \frac u{\sigma^2}\right\} \\ &\le \exp\left\{-2B_5n^{1/3}u^{2/3}\log\(\frac u{n\sigma^3}\)\right\}, \endalign $$ since $nu\ge \bar B n^{1/3}u^{2/3}\ge\bar B n\sigma^2$, and the estimate (3.4) is applicable if $\bar B$ is sufficiently large. The above estimates imply the statement of the example. \beginsection 9. Some results about $U$-statistics This section contains the proof of an important result about $U$-statistics, the so-called Hoeffding decomposition theorem which states that all $U$-statistics can be represented as a sum of degenerate $U$-statistics. Let us consider the kernel function of a $U$-statistic together with the kernel functions of the $U$-statistics in its Hoeffding decomposition. It will also be shown that the $L_2$-norm of the kernel functions of the $U$-statistics in the Hoeffding decomposition are bounded by the $L_2$-norm of the kernel function of the original $U$-statistic. Beside this, if a class of $U$-statistics is given with an $L_2$-dense class of kernel functions (with the same underlying sequence of independent and identically distributed random variables) and the Hoeffding decomposition of all of these $U$-statistics is taken, then the kernel functions of the degenerate $U$-statistics taking part in the Hoeffding decomposition also constitute an $L_2$-dense class. Another important result of this section is a decomposition of a $k$-fold random integral with respect to a normalized empirical measure to the linear combination of degenerate $U$-statistics presented in Theorem~9.4. These results enable us to prove the equivalence of Theorem~8.1 with Theorem 8.3 and of Theorem~8.2 with Theorem~8.4. They are also useful in the proof of Theorems~8.3 and~8.4. In the special case $k=1$ Hoeffding's decomposition means that the sum $S_n=\summ_{j=1}^n\xi_j$ of iid. random variables can be rewritten as $S_n=\summ_{j=1}^n(\xi_j-E\xi_j)+\(\summ_{j=1}^nE\xi_j\)$, i.e. the sum of independent random variables with zero expectation plus a constant. We may consider a constant as a $U$-statistic of order zero. (For the sake of a simpler terminology in the sequel let us consider a constant as a degenerate $U$-statistic of order zero, and define $I_{n,0}(c)=c$ for a constant $c$.) I wrote down this trivial calculation, because Hoeffding's decomposition is actually an adaptation of this procedure to the general case. To understand this let us see how to adapt this construction in the case $k=2$. In this case we have to consider a sum of the form $I_{n,2}(f)=\summ_{1\le j,k\le n,j\neq k} f(\xi_j,\xi_k)$. Write $f(\xi_j,\xi_k)=[f(\xi_j,\xi_k)-Ef(\xi_j,\xi_k|\xi_k)]+ Ef(\xi_j,\xi_k|\xi_k)=f_1(\xi_j,\xi_k)+\bar f_1(\xi_k)$ with $f_1(\xi_j,\xi_k)=f(\xi_j,\xi_k)-Ef(\xi_j,\xi_k|\xi_k)$, and $\bar f_1(\xi_k)=Ef(\xi_j,\xi_k|\xi_k)$ to achieve that the conditional expectation of $f_1(\xi_j,\xi_k)$ for fixed $\xi_k$ be zero. Repeating this procedure for the first coordinate we define $f_2(\xi_j,\xi_k)=f_1(\xi_j,\xi_k) -Ef_1(\xi_j,\xi_k|\xi_j)$ and $\bar f_2(\xi_j)= Ef_1(\xi_j,\xi_k|\xi_j)$. Simple calculation shows that $I_{n,2}(f_2)$ is a degenerate $U$-statistics of order 2, and the identity $I_{n,2}(f)=I_{n,2}(f_2)+I_{n,1}((n-1)(\bar f_1-E\bar f_1))+ I_{n,1}((n-1)((\bar f_2-E\bar f_2))+n(n-1)E(\bar f_1+\bar f_2)$ yields the decomposition of $I_{n,2}(f)$ for sums of degenerate $U$-statistics. We get the Hoeffding decomposition by working out the details of the above argument in the general case. But it is simpler to calculate the appropriate conditional expectations with the help of the kernel functions of the $U$-statistics. To carry out such a program in the study of $U$-statistics of order~$k$ we introduce the following notations. Let us consider the $k$-fold product $(X^k,\Cal X^k,\mu^k)$ of a measure space $(X,\Cal X,\mu)$ with some probability measure $\mu$, and define for all integrable functions $f(x_1,\dots,x_k)$ and indices $1\le j\le k$ the projection~$P_jf$ of the function $f$ to its $j$-th coordinate as $$ P_jf(x_1,\dots,x_{j-1},x_{j+1},\dots,x_k)=\int f(x_1,\dots,x_k)\mu(\,dx_j), \quad 1\le j\le k. \tag9.1 $$ Let us also define the operators $Q_j=I-P_j$ as $Q_jf=f-P_jf$ on the space of integrable functions on $(X^k,\Cal X^k,\mu^k)$, $1\le j\le k$. In the definition (9.1) $P_jf$ is a function not depending on the coordinate $x_j$, but in the definition of $Q_j$ we introduce the fictive coordinate $x_j$ to make the expression $Q_jf=f-P_jf$ meaningful. Now we can formulate the following result. \medskip\noindent {\bf Theorem 9.1 (Hoeffding decomposition).} {\it Let $f(x_1,\dots,x_k)$ be an integrable function on the $k$-fold product space $(X^k,\Cal X^k,\mu^k)$ of a space $(X,\Cal X,\mu)$ with a probability measure $\mu$. It has the decomposition $$ f=\summ_{V\subset\{1,\dots,k\}} f_V, \quad \text{with} \quad f_V(x_j,\,j\in V)=\(\prod_{j\in\{1,\dots,k\}\setminus V}P_j \prod_{j\in V}Q_j\)f(x_1,\dots,x_k) \tag9.2 $$ such that all functions $f_V$, $V\subset \{1,\dots,k\}$, in (9.2) are canonical with respect to the probability measure $\mu$, and they depend on the $|V|$ arguments $x_j$, $j\in V$. Let $\xi_1,\dots,\xi_n$ be a sequence of independent $\mu$ distributed random variables, and consider the $U$-statistics $I_{n,k}(f)$ and $I_{n,|V|}(f_V)$ corresponding to the kernel functions $f$, $f_V$ defined in (9.2) and random variables $\xi_1,\dots,\xi_n$. Then $$ I_{n,k}(f)=\summ_{V\subset\{1,\dots,k\}} (n-|V|)(n-|V|-1)\cdots(n-k+1)\frac{|V|!}{k!} I_{n,|V|}(f_V) \tag9.3 $$ is a representation of $I_{n,k}(f)$ as a sum of degenerate $U$-statistics, where $|V|$ denotes the cardinality of the set $V$. (The product $(n-|V|)(n-|V|-1)\cdots(n-k+1)$ is defined as 1 for $V=\{1,\dots,k\}$, i.e. if $|V|=k$.) This representation is called the Hoeffding decomposition of $I_{n,k}(f)$.} \medskip\noindent {\it The proof of Theorem 9.1.}\/ Write $f=\prodd_{j=1}^k(P_j+Q_j)f$. By carrying out the multiplications in this identity and applying the commutativity of the operators $P_j$ and $Q_j$ for different indices $j$ we get formula (9.2). To show that the functions $f_V$ in formula (9.2) are canonical let us observe that this property can be rewritten in the form $P_jf_V=0$ (in all coordinates $x_s$, $s\in V\setminus\{j\}$ if $j\in V$). Since $P_j=P_j^2$, and the identity $P_jQ_j=P_j-P_j^2=0$ holds for all $j\in\{1,\dots,k\}$ this relation follows from the above mentioned commutativity of the operators $P_j$ and $Q_j$, as $P_jf_V= \(\prodd_{s\in\{1,\dots,k\}\setminus V}P_s\prodd_{s\in V\setminus \{j\}}Q_s\)P_jQ_jf=0$. By applying the identity (9.2) for all terms $f(\xi_{j_1},\dots,\xi_{j_k})$ in the sum defining the $U$-statistic $I_{n,k}(f)$ and then summing them up we get relation (9.3). \medskip The next result enables us to estimate the kernel functions of the degenerate $U$-statistics in the Hoeffding-decomposition of a $U$-statistic by means of the properties kernel function of the original $U$-statistic. \medskip\noindent {\bf Theorem 9.2.} {\it Let $f(x_1,\dots,x_k)$ be a square integrable function on the $k$-fold product space $(X^k,\Cal X^k,\mu^k)$, and take its decomposition defined in formula (9.2). The inequalities $$ \int f_V^2(x_j,\,j\in V) \prodd_{j\in V}\mu(\,dx_j)\le \int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k) \tag9.4 $$ and $$ \sup_{x_j,\, j\in V} |f_V(x_j,\,j\in V)|\le2^{|V|}\sup_{x_j,\,1\le j\le k}|f(x_1,\dots,x_k)| \tag$9.4'$ $$ hold for all $V\subset\{1,\dots,k\}$. Let us consider an $L_2$-dense class $\Cal F$ of functions with parameter $D$ and exponent $L$ on the space $(X^k,\Cal X^k)$, take the decomposition (9.2) of all functions $f\in \Cal F$ and define the classes of functions $\Cal F_V=\{2^{-|V|}f_V\: f\in \Cal F\}$ for all $V\subset\{1,\dots,k\}$ with the help of the functions $f_V$ taking part in this decomposition. The classes of functions $\Cal F_V$ are also $L_2$-dense with the same parameter~$D$ and exponent~$L$ for all $V\subset\{1,\dots,k\}$.} \medskip Theorem 9.2 is a fairly simple consequence of Proposition~9.3 presented below. To formulate it first we introduce the following notations: Let us consider the product $(Y\times Z,\Cal Y\times\Cal Z)$ of two measurable spaces $(Y,\Cal Y)$ and $(Z,\Cal Z)$ together with a probability measure $\mu$ on $(Z,\Cal Z)$ and the operator $$ Pf(y)=P_\mu f(y)=\int f(y,z)\mu(\,dz),\quad y\in Y,\; z\in Z \tag9.5 $$ for all measurable functions $f$ on the space $Y\times Z$ for which this integral is finite. Let $I$ denote the identity operator on the space of functions on $Y\times Z$, i.e. let $If(y,z)=f(y,z)$, and introduce the operator $Q=Q_\mu=I-P=I-P_\mu$ which maps the functions $f$ on the space $Y\times Z$ to functions on the space $Y\times Z$ given by the formula $$ \aligned Q_\mu f(y,z)=(I-P_\mu)f(y,z)=f(y,z)-P_\mu f(y,z)&=f(y,z)- \int f(y,z)\mu(\,dz),\\ &\qquad y\in Y,\; z\in Z. \endaligned \tag9.6 $$ (Here, and in the sequel we shall sometimes identify a function $g(y)$ defined on the space $(Y,\Cal Y)$ with the function $\bar g(y,z)=g(y)$ on the space $(Y\times Z,\Cal Y\times \Cal Z)$ which actually does not depend on the coordinate $z$.) The following result will be proved: \medskip\noindent {\bf Proposition 9.3.} {\it Let us consider the direct product $(Y\times Z,\Cal Y\times\Cal Z)$ of two measure spaces $(Y,\Cal Y)$ and $(Z,\Cal Z)$ together with a probability measure $\mu$ on the space $(Z,\Cal Z)$. Take the transformations $P_\mu$ and $Q_\mu$ defined in formulas (9.5) and (9.6). Given any probability measure $\rho$ on the space $(Y,\Cal Y)$ consider the product measure $\rho\times\mu$ on $(Y\times Z,\Cal Y\times\Cal Z)$. Then the transformations $P_\mu$ and $Q_\mu$, as maps from the space $L_2(Y\times Z,\Cal Y\times\Cal Z,\mu\times \rho)$ to $L_2(Y,\Cal Y,\rho)$ and $L_2(Y\times Z,\Cal Y\times\Cal Z,\rho\times\mu)$ respectively, have a norm less than or equal to 1, i.e. $$ \int P_\mu f(y)^2\rho(\,dy)\le\int f(y,z)^2\rho(\,dy)\mu(\,dz), \tag9.7 $$ and $$ \int Q_\mu f(y,z)^2\rho(\,dy)\mu(\,dz)\le\int f(y,z)^2 \rho(\,dy)\mu(\,dz) \tag9.8 $$ for all functions $f\in L_2(Y\times Z,\Cal Y\times\Cal Z,\rho\times \mu)$. If $\Cal F$ is an $L_2$-dense class of functions $f(y,z)$ in the product space $(Y\times Z,\Cal Y\times\Cal Z)$, with parameter $D$ and exponent $L$, then also the classes $\Cal F_\mu=\{P_\mu f,\; f\in \Cal F\}$ and $\Cal G_\mu=\{\frac12Q_\mu f=\frac12(f-P_\mu f),\; f\in\Cal F\}$ are $L_2$-dense classes with the same exponent $L$ and parameter~$D$ in the spaces $(Y,\Cal Y)$ and $(Y\times Z,\Cal Y\times\Cal Z)$ respectively.} \medskip The following corollary of Proposition 9.3 is formally more general, but it is a simple consequence of this result. Actually we shall need this corollary. \medskip\noindent {\bf Corollary of Proposition 9.3.} {\it Let us consider the product space $(Y_1\times Z\times Y_2,\Cal Y_1\times\Cal Z\times\Cal Y_2)$, a probability measure $\mu$ on the space $(Z,\Cal Z)$ and define the transformations $$ P_\mu f(y_1,y_2)=\int f(y_1,z,y_2)\mu(\,dz),\quad y_1\in Y_1,\; z\in Z,\; y_2\in Y_2 \tag$9.5'$ $$ and $$ \aligned Q_\mu f(y_1,z,y_2)&=(I-P_\mu)f(y_1,z,y_2)=f(y_1,z,y_2)-P_\mu f(y_1,z,y_2) \\ &=f(y_1,z,y_2)-\int f(y_1,z,y_2)\mu(\,dz), \quad y_1\in Y_1,\; z\in Z, \;y_2\in Y_2 \endaligned \tag$9.6'$ $$ for the measurable functions $f$ on the space $Y_1\times Z\times Y_2$. Then $$ \int P_\mu f(y_1,y_2)^2\rho(\,dy_1,\,dy_2) \le\int f(y,z)^2(\rho\times \mu)(\,dy_1,\,dz,\,dy_2), \tag$9.7'$ $$ for all probability measures $\rho$ on $(Y_1\times Y_2,\Cal Y_1\times\Cal Y_2)$, where $\rho\times\mu$ is the product of the probability measure $\rho$ on $(Y_1\times Y_2,\Cal Y_1\times\Cal Y_2)$ and $\mu$ on $(Z,\Cal Z)$, i.e. $\rho\times\mu(\{y_1,z,y_2)\: (y_1,y_2)\in A, z\in B\})=\rho(A)\mu(B)$ for all $A\in \Cal Y_1\times\Cal Y_2$, $B\in \Cal Z$, and $\rho\times\mu$ is its unique extension as a probability measure on $(Y_1\times Z \times Y_2, \Cal Y_1\times\Cal Z\times\Cal Y_2)$. Also the inequality $$ \int Q_\mu f(y_1,z,y_2)^2\rho(\,dy_1,\,dy_2)\mu(\,dz)\le\int f(y_1,z,y_2)^2\rho(\,dy_1,\,dy_2)\mu(\,dz) \tag$9.8'$ $$ holds for all functions $f\in L_2(Y\times Z,\Cal Y\times\Cal Z, \rho\times\mu)$. If $\Cal F$ is an $L_2$-dense class of functions $f(y_1,z,y_2)$ in the product space $(Y_1\times Z\times Y_2,\Cal Y_1\times\Cal Z \times Y_2)$, with parameter $D$ and exponent $L$, then also the classes $\Cal F_\mu=\{P_\mu f,\; f\in \Cal F\}$ and $\Cal G_\mu =\{\frac12Q_\mu f=\frac12(f-P_\mu f),\; f\in\Cal F\}$ are $L_2$-dense classes with exponent $L$ and parameter~$D$ in the spaces $(Y_1\times Y_2,\Cal Y_1\times \Cal Y_2)$ and $(Y_1\times Z\times Y_2,\Cal Y_1\times\Cal Z\times\Cal Y_2)$ respectively.} \medskip This corollary is a simple consequence of Proposition~9.3 if we apply it with $(Y,\Cal Y)=(Y_1\times Y_2,\Cal Y_1\times\Cal Y_2)$ and take the natural mapping $f((y_1,y_2),z)\to f(y_1,z,y_2)$ of a function from the space $(Y\times Z,\Cal Y\times \Cal Z)$ to a function on $(Y_1\times Z\times Y_2,\Cal Y_1\times\Cal Z\times \Cal Y_2)$, and use the correspondence between the product measure $\rho\times \mu$ in these spaces. Proposition 9.3, more precisely its corollary implies Theorem 9.2, since it implies that the operators $P_s$, $Q_s$, $1\le s\le k$, applied in Theorem~9.2 do not increase the $L_2(\mu)$ norm of a function $f$, and it is also clear that the norm of $P_s$ is bounded by 1 the norm of $Q_s=I-P_s$ is bounded by 2 as an operator from $L_\infty$ spaces to $L_\infty$ spaces. The corollary of Proposition~9.3 also implies that if $\Cal F$ is an $L_2$-dense class of functions with parameter $D$ and exponent~$L$, then the same property holds for the classes of functions $\Cal F_{P_s}=\{P_sf\: f\in \Cal F\}$ and $\Cal F_{Q_s}=\{\frac12O_sf\: f\in \Cal F\}$, $1\le s\le k$. These relations together with the identity $f_V=\(\prodd_{s\in V}P_s\prodd_{s\in\{1,\dots,k\}\setminus V}Q_s\)f$ imply Theorem~9.2. \medskip\noindent {\it Proof of Proposition 9.3.}\/ The Schwarz inequality yields that $P_\mu(f)^2\le\int f(y,z)^2\mu(\,dz)$, and integrating this inequality with respect to the probability measure $\rho(\,dy)$ we get inequality (9.7). Also the inequality $$ \int Q_\mu f(y,z)^2\rho(dy)\mu(dz)=\int [f(y,z)-P_\mu f(y,z)]^2\rho(du)\mu(dz) \le\int f(y,z)^2\rho(dy)\mu(dz) $$ holds, and this is relation (9.8). It follows for instance from the observation that the functions $f(y,z)-P_\mu f(y,z)$ and $P_\mu f(y,z)$ are orthogonal in the space $L_2(Y\times Z,\Cal Y\times\Cal Z,\rho\times\mu)$. Let us consider an arbitrary probability measure $\rho$ on the space $(Y,\Cal Y)$. To prove that $\Cal F_\mu$ is an $L_2$-dense class with parameter~$D$ and exponent~$L$ we have to find $m\le D \e^L$ functions $f_j\in \Cal F_\mu$, $1\le j\le m$, such that $\inff_{1\le j\le m}\int (f_j-f)^2\,d\rho\le \e^2$ for all $f\in \Cal F_\mu$. But a similar property holds in the space $Y\times Z$ with the probability measure $\rho\times\mu$. This property together with the $L_2$ contraction property of $P_\mu$ formulated in (9.7) imply that $\Cal F_\mu$ is an $L_2$-dense class. To prove that $\Cal G_\mu$ is also $L_2$-dense with parameter $D$ and exponent $L$ we have to find for all numbers $0<\e\le1$ and probability measures $\rho$ on $Y\times Z$ a subset $\{g_1,\dots,g_m\}\subset \Cal G_\mu$ with $m\le D\e^{-L}$ elements such that $\inff_{1\le j\le m}\int (g_j-g)^2\,d\rho\le \e^2$ for all $g\in \Cal G_\mu$. Let us consider the probability measure $\tilde\rho=\frac12(\rho+\bar\rho\times\mu)$ on $(Y\times Z,\Cal Y\times\Cal Z)$, where $\bar\rho$ is the projection of the measure $\rho$ to $(Y,\Cal Y)$, i.e. $\bar\rho(A)=\rho(A\times Z)$ for all $A\in\Cal Y$, take a class of function $\Cal F_0(\e,\tilde \rho) =\{f_1,\dots,f_m\}\in\Cal F$, $m\le D\e^{-L}$ such that $\inff_{1\le j\le m}\int (f_j-f)^2\,d\tilde\rho\le \e^2$ for all $f\in\Cal F$, and put $\{g_1,\dots,g_m\}=\{\frac12Q_\mu f_1,\dots, \frac12Q_\mu f_m\}$. All functions $g\in\Cal G_\mu$ can be written in the form $g=\frac12Q_\mu f$ with some $f\in \Cal F$, and there exists some function $f_j\in\Cal F_0(\e,\tilde\rho)$ such that $\int (f-f_m)^2\,d\tilde\rho\le\e^2$. Hence to complete the proof of Proposition~9.3 it is enough to show that $\int\frac14(Q_\mu f -Q_\mu\bar f)^2\,d\rho\le\int(f-\bar f)^2\,d\tilde\rho$ for all pairs $f,\bar f\in\Cal F$. This inequality holds, since $\int\frac14(Q_\mu f-Q_\mu\bar f)^2\,d\rho\le\int\frac12(f-\bar f)^2\,d\rho+\int\frac12(P_\mu f-P_\mu\bar f)^2\,d\rho$, and $\int(P_\mu f-P_\mu\bar f)^2\,d\rho=\int(P_\mu f-P_\mu\bar f)^2\,d\bar\rho\le\int(f-\bar f)^2\,d(\bar\rho\times\mu)$ by formula 9.7. The above relations imply that $\int\frac14(Q_\mu f-Q\mu\bar f)^2\,d\rho\le \int(f-\bar f)^2\frac12 d\,(\rho+\bar\rho\times\mu)=\int(f-\bar f)^2d\,\tilde\rho$ as we have claimed. Let us turn to the proof of the equivalence of Theorem~$8.1'$ with 8.3 and of Theorem 8.2 with 8.4. In Theorems~8.2 and~8.4 we can restrict our attention to the case when the class of functions $\Cal F$ is countable, since the case of countably approximable classes can be simply reduced to this situation. Let us remark that the integration with respect to the measure $\mu_n-\mu$ in the definition (4.8) of the integral $J_{n,k}(f)$ means some kind of normalization, and no such normalization appears in the definition of the $U$-statistics $I_{n,k}(f)$. This is the cause why degenerate $U$-statistics had to be considered in Theorems~8.3 and~8.4. The deduction of these results from Theorems~$8.1'$ and~8.2 is fairly simple if the underlying probability measure $\mu$ is non-atomic, since in this case the identity $I_{n,k}(f)=J_{n,k}(f)$ holds for a canonical function with respect to the measure $\mu$. Let us remark that the non-atomic property of the measure $\mu$ is needed in this argument not only because of the conditions of Theorems~$8.1'$ and~8.2, but since in the proof of the above relation we need the identity $\int f(x_1,\dots,x_k)\mu(\,dx_j)=0$ in the case when the domain of integration is a set of the form $X\setminus\{x_1,\dots,x_{j-1},x_{j+1},\dots,x_k\}$. The case of possibly atomic measures $\mu$ can be simply reduced to the case of non-atomic measures by means of the following enlargement of the space $(X,\Cal X,\mu)$. Let us introduce the product space $(\bar X,\bar{\Cal X},\bar\mu)=(X,\Cal X,\mu) \times([0,1],\Cal B,\lambda)$, where $\Cal B$ is the $\sigma$-algebra and $\lambda$ is the Lebesgue measure on $[0,1]$. Define the function $\bar f((x_1,u_1),\dots,(x_k,u_k))=f(x_1,\dots,x_k)$ on this enlarged space. Then $I_{n,k}(f)=I_{n,k}(\bar f)$, and the measure $\bar\mu=\mu\times\lambda$ is non-atomic. Hence we can deduce the estimates of Theorems~8.3 and~8.4 from Theorems~$8.1'$ and~8.2 by deducing them first for their counterpart in the above constructed enlarged space and the above defined functions. The deduction of Theorems~$8.1'$ and~8.2 from Theorems~8.3 and~8.4 requires more work. Let us observe that an integral $J_{n,k}(f)$ can be written as a sum of $U$-statistics of different order, and by applying the Hoeffding decomposition for each term in this sum we can express the integral $J_{n,k}(f)$ as a sum of degenerate $U$-statistics. We show that the coefficients of the degenerate $U$-statistics in the above representation have relatively small coefficients. This is the content of the following Theorem~9.4. To make its content more understandable I formulated its main statement in the case of random integrals of multiplicity two in a more explicit form. \medskip\noindent {\bf Theorem 9.4.} {\it Let us have a non-atomic measure $\mu$ on a measurable space $(X,\Cal X)$ together with a sequence of independent, $\mu$-distributed random variables $\xi_1,\dots,\xi_n$, and take a function $f(x_1,\dots,x_k)$ of $k$ variables on the space $(X^k,\Cal X^k)$ such that $$ \int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)<\infty. $$ Let us consider the empirical distribution function $\mu_n$ of the sequence $\xi_1,\dots,\xi_n$ introduced in (4.5) together with the $k$-fold random integral $J_{n,k}(f)$ of the function $f$ defined in (4.8). The identity $$ J_{n,k}(f)=\sum_{V\subset\{1,\dots,k\}}C(n,k,V)n^{-|V|/2} I_{n,|V|}(f_V), \tag9.9 $$ holds with the set of (canonical) functions $f_V(x_j,\;j\in V)$ (with respect to the measure $\mu$) defined in formula (9.2) together with some real numbers $C(n,k,V)$, $V\subset\{1,\dots,k\}$, where $I_{n,|V|}(f_V)$ denotes the (degenerate) $U$-statistic of order $|V|$ with the random variables $\xi_1,\dots,\xi_n$ and kernel function $f_V$. The constants $C(n,k,V)$ in formula (9.9) satisfy the inequality $|C(n,k,V)|\le C(k)$ with some constant $C(k)$ depending only on the order $k$ of the integral $J_{n,k}(f)$. The relations $\limm_{n\to\infty}C(n,k,V)=C(k,V)$ with some appropriate constant such that $0\le |C(k,V)|<\infty$ and $C(n,k,\{1,\dots,k\})=1$ for $V=\{1,\dots,k\}$ also hold.} \medskip\noindent {\it Remark:}\/ Some considerations show that the coefficients $C(n,k,V)$ in formula (9.9) depend only on the cardinality $|V|$ of the set $V$, i.e. we can write $C(n,k,V)=C(n,k,|V|)$. We shall not need this observation. \medskip Theorems~$8.1'$ and~8.2 can be simply deduced from Theorems~8.3 and~8.4 respectively with the help of Theorem~9.4. Indeed, to deduce Theorem~$8.1'$ we can write with the help of formula~9.9 $$ P(|J_{n,k}(f)|>u)\le \sum_{V\subset\{1,\dots,k\}} P\(n^{-|V|/2}|I_{n,|V|}(f_V)|>\frac u{2^kC(k)}\) \tag9.10 $$ with a constant $C(k)$ satisfying the inequality $C(n,k,|V|)\le C(k)$ for all coefficients $C(n,k,|V|)$ in~(9.9). Then we get Theorem~$8.1'$ from Theorem~8.3 and relations~(9.4) and~$(9.4')$ in Theorem~9.2 by which the $L_2$-norm of the functions $f_V$ are bounded by the $L_2$-norm of the function~$f$ and the $L_\infty$-norm of $f_V$ is bounded by the $2^{|V|}$-times the $L_\infty$-norm or $f$ if we estimate each term at the right-hand side of (9.10) by means of Theorem~8.3. Here we may assume that $2^kC(k)>1$ and let us first assume that also the inequality $\frac u{2^kC(k) \sigma}\ge1$ holds. In this case we get formula $(8.3')$ in Theorem~$8.1'$ by estimating each term at the right-hand side of (9.10). Observe that $\exp\left\{-\alpha\(\frac u{2^kC(k)\sigma}\)^{2/s}\right\} \le \exp\left\{-\alpha\(\frac u{2^kC(k)\sigma}\)^{2/k}\right\}$ for all $s\le k$ if $\frac u{2^kC(k)\sigma}\ge1$. If $\frac u{2^kC(k)\sigma}\le1$, then formula $(8.3')$ holds with a sufficiently large $C>0$. Theorem~8.2 can be similarly deduced from Theorem~8.4 if we observe that relation (9.10) remains valid if we replace $|J_{n,k}(f)|$ by $\supp_{f\in\Cal F}|J_{n,k}(f)|$ and $|I_{n,|V|}(f_V)|$ by $\supp_{f_V\in\Cal F_V} |I_{n,|V|}(f_V)|$ in it, and the constant~$M$ in formula~(8.6) of Theorem~8.2 is chosen sufficiently large. The only difference is that now we have to exploit beside formulas (9.4) and~$(9.4')$ of Theorem~9.2 the last statement of this result which tells that if $\Cal F$ is an $L_2$-dense class of functions on a space $(X^k,\Cal X^k)$, then the classes of functions $\Cal F_V= \{2^{-|V|}f_V\:f\in\Cal F\}$ are also $L_2$-dense classes of functions for all $V\subset\{1,\dots,k\}$ with the same exponent and parameter. In the definition of the random integrals $J_{n,k}(f)$ we have integrated in all coordinates with respect to the signed measure $\mu_n-\mu$, and this means some kind of normalization. Thus it is not surprising that the tail behaviour of the distribution of $J_{n,k}(f)$ is similar to that of certain degenerate $U$-statistics. Theorem~9.4 formulates such a relation. Formula~(9.9) expresses the random integral $J_{n,k}(f)$ as a linear combination of degenerate $U$-statistics of different order. It is similar to the Hoeffding decomposition in that respect that the functions $f_V$ in formula (9.9) agree with the functions $f_V$ appearing in the Hoeffding decomposition of the $U$-statistic $I_{n,k}(f)$ with kernel function $f$. But the coefficients in the expansion (9.9) are small. On the other hand, these coefficients need not disappear. In particular, the expansion (9.9) may contain a non-zero constant term. In such a case the expected value $EJ_{n,k}(f)$ may not equal zero, but it can be bounded by a number not depending on the sample size~$n$. In the next example I show that there are really random integrals $J_{n,k}(f)$ such that $EJ_{n,k}(f)\neq0$. Let us choose a sequence of independent random variables $\xi_1,\dots,\xi_n$ with uniform distribution on the unit interval, let $\mu_n$ denote its empirical distribution, let $f=f(x,y)$ denote the indicator function of the unit square, i.e. let $f(x,y)=1$ if $0\le x,y\le1$, and $f(x,y)=0$ otherwise. Let us consider the random integral $J_{n,2}(f)=n\int_{x\neq y} f(x,y) (\mu_n(\,dx)-\,dx)(\mu_n(\,dy)-dy)$, and calculate its expected value $EJ_{n,2}(f)$. By adjusting the diagonal $x=y$ to the domain of integration and taking out the contribution obtained in this way we get that $EJ_{n,2}(f)=nE(\int_0^1\(\mu_n(\,dx)-\mu(\,dx)\)^2 -n^2\cdot\frac1{n^2}=-1$. (The last term is the integral of the function $f(x,y)$ on the diagonal $x=y$ with respect to the product measure $\mu_n\times\mu_n$ which equals $(\mu_n-\mu)\times(\mu_n-\mu)$ on the diagonal.) The above considerations and the proof of Theorem~9.4 indicate that the equivalence between Theorems~$8.1'$ and~8.3 or between Theorems~8.2 and~8.4 is not self-evident. It is simpler to prove Theorems~8.3 and~8.4 of these theorem pairs about degenerate $U$-statistics, and this will be done in this work. On the other hand, Theorems~$8.1'$ and~8.2 seem to be more appropriate for applications, since here we do not have to restrict our attention to special, canonical kernel functions. \medskip\noindent {\it The proof of Theorem 9.4.}\/ Let us first introduce the (random) probability measures $\mu^{(l)}$, $1\le l\le n$, concentrated in the sample points $\xi_l$, i.e. let $\mu^{(l)}(A)=1$ if $\xi_l\in A$, and $\mu^{(l)}(A)=0$ if $\xi_l\notin A$, $A\in \Cal A$. Then $\mu_n-\mu=\frac1n\(\summ_{l=1}^n\(\mu^{(l)}-\mu\)\)$, and formula (4.8) can be rewritten as $$ \align J_{n,k}(f)=\dfrac1{n^{k/2}k!}&\sum_{(l_1,\dots,l_k),\, 1\le l_j\le n, \,1\le j\le k} \int' f(x_1,\dots,x_k) \tag9.11 \\ &\qquad \(\mu^{(l_1)}(\,dx_1)-\mu(\,dx_1)\)\dots \(\mu^{(l_k)}(\,dx_k)-\mu(\,dx_k)\). \endalign $$ To rearrange the above sum in a way more appropriate for us let us introduce the class of all partitions $\Cal P=\Cal P_k$ of the set $\{1,2,\dots,k\}$. For a partition $P=\{R_1,\dots,R_u\}$ $\bigcupp_{j=1}^u R_j=\{1,\dots,k\}$, $R_j\cap R_l=\emptyset$, $1\le j0$. Beside this, it is not difficult to see that the identity $C(n,k,\{1,\dots,k\})=1$ holds. To see that the estimate $|C(n,k,V)|\le C(k)$ really holds, observe that $n^{-|V|/2}C(n,k,|V|)$ can be written as a sum of finitely many terms, (the number of terms can be bounded by a number depending only on~$k$) such that all of them can be bounded by a number of the form $D(k) n^{-k/2+\beta(P,V)}$ with some partition $P$ and the number $\beta(P,V)$ introduced after formula (9.16) with some $P\in \Cal P_k$ and $V\in\bar{\Cal T}(P)$. Hence it is enough to show that $-\frac k2+\beta(P,V)\le-\frac{|V|}2$, i.e. $\beta(P,V)\le \frac{k-|V|}2$ if $V\in\bar{\Cal T}(P)$. This relation clearly holds, since $\beta(P,V)$ is the number of components of a partition of a set with cardinality less than or equal to $k-|V|$, and all components of this partition have a cardinality at least~2. Relation (9.18) can be rewritten as $J_{n,k}(f)=\!\!\!\! \summ_{V\subset \{1,2,\dots,k\}}\!\!\!\! C(n,k,V) n^{-|V|/2} I_{n,|V|}(f_V)$, where $I_{n,|V|}(f_V)$ is the $U$-statistic with the random variables $\xi_1,\dots,\xi_n$ and the kernel function $f_V$ defined in (9.19) agrees with the function $f_V$ defined in (9.2). We have also seen that the coefficients $C(n,k,V)$ satisfy the inequality stated in~Theorem~9.4. Relation (9.17) together with the bound on the terms $\beta(P,V)$ also imply that the finite limits $\limm_{n\to\infty}C(n,k,V)=C(k,V)$ also exist. Theorem~9.4 is proved. \medskip I formulate two corollaries of Theorem~9.4. The first one explains the content of conditions (8.2) and (8.5) in Theorems~8.1---8.4. \medskip\noindent {\bf Corollary~1 of Theorem 9.4.}\/ {\it If $I_{n,k}(f)$ is a degenerate $U$-statistic of order $k$ with some kernel function $f$, then $E\(n^{-k/2}I_{n,k}(f)\)^2\le\frac1{k!}\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)$, where $\mu$ is the distribution of the random variables taking part in the definition of the $U$-statistic~$I_{n,k}(f)$. Analogously, the $k$-fold multiple random integral $J_{k,n}(f)$ satisfies the inequality $E\(n^{-k/2}J_{n,k}(f)\)^2\le\bar C(k)\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)$ with some constant $\bar C(k)$ depending only on the order~$k$ of the integral~$J_{n,k}(f)$.} \medskip\noindent {\it Proof of Corollary~1 of Theorem 9.4.} We have $$ E(n^{-k/2}I_{n,k}(f))^2=\frac1{(k!)^2n^k}\sum{\vphantom \sum}' Ef(\xi_{l_1},\dots,\xi_{\l_s})f(\xi_{l_1'},\dots,\xi_{\l_s'}), $$ where the prime in $\sum'$ means that summation is taken for such pairs of $k$-tuples $(l_1,\dots,l_k)$, $(l_1',\dots,l_k')$, $1\le l_j,l_j'\le n$, for which $l_j\neq l_{j'}$ and $l'_j\neq l_{j'}'$ if $j\neq j'$. The degeneracy of the $U$-statistic $I_{n,k}(f)$ implies that $Ef(\xi_{l_1},\dots,\xi_{\l_s})f(\xi_{l_1'},\dots,\xi_{\l_s'})=0$ if the two $k$-tuples $(l_1,\dots,\l_s)$ and $(l_1',\dots,\l_s')$ differ. This can be seen by taking such an index $l_j$ from the first $k$-tuple which does not appear in the second one, and by observing that the conditional expectation of the product we consider equals zero by the degeneracy condition of the $U$-statistic under the condition that the value of all random variables except that of $\xi_{l_j}$ is fixed in this product. There remains $k!n(n-1)\cdots (n-k+1)$ terms in the sum expressing $E\(n^{-k/2}I_{n,k}(f)\)^2$ which may be non-zero, and all of them can be bounded by $\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)$ because of our conditions and the Schwarz inequality. These estimates yield the bound given for $E\(n^{-k/2}I_{n,k}(f)\)^2$. We can simply get the bound for $J_{n,k}(f)$ with the help of Theorem~9.4, formula (9.4) in Theorem 9.4 by which the $L_2$-norm of the functions $f_V$ can be bounded by the $L_2$-norm of the function $f$ and the bound given for the second moment of degenerate $U$-statistics $n^{-|V|/2}I_{n,|V|}(f_V)$ appearing in the expansion~(9.9). \medskip In Corollary~2 the decomposition (9.9) of a random integral $J_{n,2}(f)$ of order 2 is described in an explicit way. \medskip\noindent {\bf Corollary 2 of Theorem 9.4.} {\it Let the random integral $J_{n,2}(f)$ satisfy the conditions of Theorem 9.4. In this case formula (9.9) can be written in the following explicit form: $$ J_{n,2}(f)=\frac1n I_{n,2}(f_{\{1,2\}})-\frac1n I_{n,1}(f_{\{1\}}) -\frac1n I_{n,1}(f_{\{2\}})-f_\emptyset \tag$9.9'$ $$ with the functions $$ \allowdisplaybreaks \align f_{\{1,2\}}(x,y)&=f(x,y)-\int f(x,y)\mu(\,dx)- \int f(x,y)\mu(\,dy)+\int f(x,y)\mu(\,dx)\mu(\,dy), \\ f_{\{1\}}(x)&=\int f(x,y)\mu(\,dy)-\int f(x,y)\mu(\,dx)\mu(\,dy), \\ f_{\{2\}}(y)&=\int f(x,y)\mu(\,dx)-\int f(x,y)\mu(\,dx)\mu(\,dy) \endalign $$ and $f_\emptyset=\int f(x,y)\mu(\,dx)\mu(\,dy)$.} \beginsection 10. The proof of Theorem 8.3 about the distribution of $U$-statistics This section contains the proof of Theorem~8.3 about the distribution of degenerate $U$-statistics with the help of some results which are interesting in themselves. One of these results, called Borell's inequality, gives an estimate on the moments of homogeneous polynomials of Rademacher functions, another result we need is a symmetrization type estimate which can be considered as the multivariate version of the more interesting part of the Marcinkiewicz--Zygmund inequality. Finally there is a third result we apply which compares the distribution of a $U$-statistics with the distribution of an appropriate modification of it. The first two results will be proved in the next section, the third one in the Appendix. Theorem~8.3 can be considered as the generalization of Bernstein's inequality (Theorem~3.1) for~$U$-statistics. Bernstein's inequality was proved by means of an estimation of the moment-generating function of the partial sums of independent and bounded random variables. This approach has to be modified in the proof of Theorem~8.3. In such cases we cannot work well with the moment generating functions, since if the sample size tends to infinity, then the normalized version of degenerate $U$-statistics of order~$k$ have a limit distribution $F$ with a tail-behaviour $1-F(x)\ge e^{-C x^{2/k}}$ with some $C>0$ as $x\to \infty$. (This is a relatively well-known result, but we shall not need it in this work.) This means that a random variable with this limit distribution has no moment generating function for $k\ge3$. On the other hand, the proof of Theorem~8.3 is relatively simple, if we have a good estimate also for the high moments of degenerate $U$-statistics. Such a moment estimate is formulated in the following \medskip\noindent {\bf Proposition 10.1.} {\it Let us consider a canonical function $f=f(x_1,\dots,x_k)$ on the $k$-fold product $(X^k,\Cal X^k,\mu^k)$ of a measure space $(X,\Cal X,\mu)$ together with a sequence of independent $\mu$ distributed random variables and the degenerate $U$-statistic $I_{n,k}(f)$ determined by this sequence of random variables $\xi_1,\dots,\xi_n$ and canonical function~$f$. Let us also assume that the function $f$ satisfies conditions~(8.1) and~(8.2) with some number~$0<\sigma\le1$. Then there exists some constants $C=C_k>0$ such that the moments of the $U$-statistic $I_{n,k}(f)$ defined in formula (8.7) satisfy the inequality $$ E\(\left|n^{-k/2}I_{n,k}(f)\right|^{2M}\)\le C^M_k M^{kM}\sigma^{2M}\quad \text{\rm if } 1\le M\le n\sigma^2. \tag10.1 $$ }\medskip Let us consider the $k$-th power of a standard normal random variable $\eta$ and calculate the asymptotic magnitude of the $2M$-th moment $E\(\sigma\eta^k\)^{2M}$ of $\sigma\eta^k$ for large~$M$. We have $E\(\sigma\eta^k\)^{2M}=1\cdot3\cdots(2kM-1)\sigma^{2M} =\frac{(2kM)!}{2^{kM}(kM)!}\sigma^{2M}\sim\(\frac{2k}e\)^{kM} M^{kM}\sigma^{2M}$ by the Stirling formula. This means that the estimate given for the $2M$-th moments of a normalized $U$-statistics $n^{-k/2}I_{n,k}(f)$ in formula (10.1) has the same order as the $2M$-th moment of the random variable $\const\sigma\eta^k$, at least if $1\le M\le n\sigma^2$. This estimate will imply Theorem~8.3 which also can be so interpreted that $P\(n^{-k/2}I_{n,k}(f)>u\)$ can be bounded by $\const P\(\const\sigma\eta^k>u\)$, at least if $00$ and $\gamma=\gamma(k)>0$ depending only on the order~$k$ of the $U$-statistic such that $$ P\(\|I_{n,k}(f)\|>u\)\le \bar CP\(\|\bar I_{n,k}(f)\|>\gamma u\) \tag10.8 $$ for all $u>0$. Here $\|\cdot\|$ denotes the norm in the Banach space~$B$ where the function~$f$ takes its values. More generally, if we have a countable sequence of functions $f_s$, $s=1,2,\dots$, taking their values in the same separable Banach-space, then $$ P\(\sup_{1\le s<\infty} \left\| I_{n,k}(f_s)\right\|>u\)\le \bar CP\(\sup_{1\le s<\infty}\left\|\bar I_{n,k}(f_s)\right\|>\gamma u\). \tag$10.8'$ $$ }\medskip We follow the following approach. We shall prove such a version of Proposition~10.1 and Theorem~8.3 where $U$-statistics are replaced by decoupled $U$-statistics. The proof of these results is simpler, because the arguments applied for $U$-statistics also work for decoupled $U$-statistics, and also Proposition~10.3 can be applied in this case. Theorem~8.3 can be obtained as a consequence of its version we shall prove and Theorem~10.4. More explicitly, we shall prove the following two results. \medskip\noindent {\bf Proposition $10.1'$.} {\it Let the conditions of Proposition~10.1 be satisfied with some sequence of iid. $\mu$-distributed random variables $\xi_1,\dots,\xi_n$ on a space $(X,\Cal X)$, a function $f$ on the product space $(X^k,\Cal X^k)$ and a number $0<\sigma\le1$. Take $k$ independent copies $\xi_1^{(j)},\dots,\xi_n^{(j)}$, $1\le j\le k$, of the random sequence $\xi_1,\dots,\xi_n$, and define with their help the decoupled $U$-statistic $I_{n,k}(f)$ defined in (10.5). Then the inequality $$ E\(\left|n^{-k/2}\bar I_{n,k}(f)\right|^{2M}\)\le C^M_k M^{kM}\sigma^{2M}\quad \text{\rm if } 1\le M\le n\sigma^2 \tag$10.1'$ $$ holds with some constant $C_k$ which depends only on the order $k$ of the decoupled $U$-statistic.} \medskip\noindent {\bf Theorem $8.3'$.} {\it Let the conditions of Proposition~8.3 be satisfied with some sequence of iid. $\mu$-distributed random variables $\xi_1,\dots,\xi_n$ on a space $(X,\Cal X)$, a function $f$ on the product space $(X^k,\Cal X^k)$ and a number $0<\sigma\le1$. Take $k$ independent copies $\xi_1^{(j)},\dots,\xi_n^{(j)}$, $1\le j\le k$, of the random sequence $\xi_1,\dots,\xi_n$, and define with their help the decoupled $U$-statistic $\bar I_{n,k}(f)$ defined in (10.5). Then there exist some constants $C=C(k)>0$ and $\alpha=\alpha(k)>0$ such that the inequality $$ P\(n^{-k/2}|\bar I_{n,k}(f)|>u\)\le C \exp\left\{-\alpha \(\frac u\sigma\)^{2/k}\right\} \tag10.9 $$ holds for all $00$, where $\e_1,\dots,\e_n$, $P(\e_l=1)=P(\e_l=-1)=\frac12$ are independent random variables, independent also of the random sequence $\xi_1,\dots,\xi_n$. Indeed, formula (10.4) implies that $E\left|\summ_{l=1}^n\e_l\xi_l\right|^p \le (p-1)^{p/2} E\(\summ_{l=1}^n\xi_l^2\)^{p/2}$. Let us also observe that Proposition~10.3 is a multivariate generalization of formula $(10.12')$ with the additional (important) information that it gives a good explicit choice for the coefficient $\bar C_p$ in it. We can prove with the help of Borell's inequality such an inequality which has similar relation to Proposition ~10.3 as the right-hand side inequality in formula (10.12) to formula $(10.12')$. We shall give this result in the following corollary, and actually we shall apply this consequence of Proposition~10.3. \medskip\noindent {\bf Corollary of Proposition 10.3.} {\it Let the conditions of Proposition~10.3 hold with the additional restriction that the inequality $E|f(\xi_1,\dots,\xi_k)|^p<\infty$ holds with some $p\ge2$ (i.e. $p>1$ is not sufficient for us). Then also the inequality $$ E|\bar I_{n,k}(f)|^p\le2^{kp} p^{kp/2} E\bar I_{n,k}(f^2)^{p/2} \tag10.13 $$ holds.} \medskip\noindent {\it Proof of the Corollary of Proposition 10.3.} Let $\Cal F$ denote the $\sigma$-algebra generated by the random variables $\xi^{(j)}_1,\dots,\xi_n^{(j)}$, $1\le j\le k$. Then Proposition~10.3 implies that $$ E|\bar I_{n,k}(f)|^p\le2^{kp} E|\bar I_{n,k}(f,\e)|^p= 2^{kp}E(E(|\bar I_{n,k}(f,\e)|^p|\Cal F)). $$ On the other hand, the consequence of Borell's inequality formulated in relation (10.4) yields that $$ \align &E(|\bar I_{n,k}(f,\e)|^p|\Cal F) =E_\e\left|\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \e_{l_1}\cdots\e_{l_k} f\(\xi_{l_1}^{(1)},\dots,\xi_{l_k}^{(k)}\)\right|^p\\ &\qquad\qquad\le p^{kp/2}\(\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb f^2\(\xi_{l_1}^{(1)},\dots,\xi_{l_k}^{(k)}\)\)^{p/2} =p^{kp/2}\bar I_{n,k}(f^2)^{p/2}, \endalign $$ where $E_\e$ means that we fix the values of the random variables $\xi^{(j)}_1,\dots,\xi_n^{(j)}$, $1\le j\le k$ and take expectation with respect to the random variables $\e_j$, $1\le j\le n$. We get, by taking expectation in the last inequality, that $E|\bar I_{n,k}(f,\e)|^p\le p^{kp/2}E\bar I_{n,k}(f^2)^{p/2}$. This inequality together with formula (10.7) imply relation (10.13). \medskip Now we turn to the proof of Proposition~$10.1'$. \medskip\noindent {\it The proof of Proposition $10.1'$.} We have $En^{-k}\bar I_{n,k}(f)^2\le \frac1{k!^2}\sigma^2$ if $f$ is a canonical function with respect to the probability measure $\mu$, and $\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)\le\sigma^2$, i.e. relation $(10.1')$ in Proposition~$10.1'$ holds for $M=1$ if $C_k\ge\frac1{k!^2}$, because $$ Ef(\xi^{(1)}_{l_1},\dots,\xi^{(k)}_{l_k}) f(\xi^{(1)}_{l_1'},\dots,\xi^{(k)}_{l_k'})=0, \quad \text{if } l_j\neq l_j' \quad\text{for some index }1\le j\le k, $$ and $Ef^2(\xi^{(1)}_{l_1},\dots,\xi^{(k)}_{l_k})\le\sigma^2$. First we prove relation $(10.1')$ in the special case $M=2^m$ with $m=0,1,\dots$ if $1\le M\le 2n\sigma^2$ and the constants $C_k$ are chosen appropriately in $(10.1')$. We have already proved this relation for $m=0$. We shall prove the inequality $E(n^{-k/2}I_{n,k}(f)^{2M}) \le C_k^M M^{kM}\sigma^{2M}$ for all $k=1,2,\dots$ with some appropriate constant $C_k>0$ if $M=2^m$ and $M\le 2n\sigma^2$ by induction with respect to $m$. In the proof formula (10.13) of the Corollary of Proposition~10.3 will be applied with the choice $p=2M$. This yields the estimate $$ E\(\(n^{-k/2}\bar I_{n,k}(f)\)^{2M}\)\le2^{2kM}(2M)^{Mk} E\(n^{-k}\bar I_{n,k}(f^2)\)^M. \tag10.14 $$ The above inequality is not sufficient in its original form to carry out the inductive procedure we have in mind, since the function $f^2$ appearing at its right-hand side is not canonical. But this difficulty can be overcome if we apply the Hoeffding decomposition (9.2) for the function $f^2$. This result yields a representation of the form $$ f^2(x_1,\dots,x_k)=\summ_{V\subset\{1,\dots,k\}} f_V(x_s,s\in V) $$ with some appropriate canonical functions $f_V(x_s,s\in V)$ with respect to the measure $\mu$ for all $V\subset\{1,\dots,k\}$. This relation implies that similarly to $U$-statistics decoupled $U$-statistics satisfy the relation $$ \bar I_{n,k}(f^2)=\summ_{V\subset\{1,\dots,k\}} (n-|V|)(n-|V|-1)\cdots(n-k+1)\frac{|V|!}{k!} \bar I_{n,|V|}(f_V). \tag10.15 $$ In Theorem 9.1 the functions $f_V$ appearing in formula (10.15) are described explicitly. (Here again we define the value of the product $(n-|V|)(n-|V|-1)\cdots(n-k+1)$ as 1 for $|V|=k$.) We do not need this formula, we only need that by formulas (9.4) and $(9.4')$ of Theorem~9.2 the integrals of the square of the functions $f_V$ are bounded by $\sigma^2$, and these functions are bounded by $2^{|V|}$ in supremum norm, because the function $f^2$, similarly to the function $f$, is bounded by $\sigma$ in $L_2(\mu)$-norm, and it is bounded by~1 in the supremum norm. The coefficient $f_V$ with $V=\emptyset$ in the constant term of the sum at the right-hand side of (10.15) has to be considered separately. It equals $f_{\emptyset}=\int f^2(x_1,\dots,x_k) \mu(\,dx_1)\dots\mu(\,dx_k)$. This implies that $0\le f_{\emptyset}\le\sigma^2$, an estimate which does not follow directly from Theorem~9.2. Formula (10.15) and the triangular inequality in $L_M$ norm imply the inequality $$ \aligned &E(n^{-k}\bar I_{n,k}(f^2))^{M}\le \(n^{-k}\sum_{V\subset\{1,\dots,k\}} \(E\(n^{k-|V|} \frac{(|V|)!}{k!}I_{n,|V|}(f_V)\)^M\)^{1/M}\)^M\\ & \qquad \le\(\sum_{j=0}^k \binom kj \frac{j!}{k!} \sup_{V\: V\subset\{1,\dots,k\},\,|V|=j} \(n^{-jM/2}E(n^{-j/2}\bar I_{n,j}(f_V))^M\)^{1/M}\)^M \!\!. \endaligned \tag10.16 $$ The function $2^{-j}|f_V|$ is bounded by 1 in the supremum norm and by $2^{-j/2}\sigma\le\sigma$ in the $L_2(\mu)$ norm if $|V|=j$, $1\le j\le k$. Our inductive hypothesis implies that the terms at the right-hand side of (10.16) can be estimated as $$ \align n^{-jM/2}E(n^{-j/2}\bar I_{n,j}(f_V))^M&\le 2^{jM} C_j^{M/2}\sigma^M \(\frac M2\)^{jM/2} n^{-jM/2}\\ &=(2^jC_j)^{M/2}\sigma^{2M} \(\frac {M}{n\sigma^{2/j}}\)^{jM/2} \le 2^{jM}C_j^{M/2}\sigma^{2M}, \\ &\qquad\qquad\text{if } |V|=j, \;1\le j\le k, \text{ and }M\le 2n\sigma^2, \endalign $$ since $\frac M{n\sigma^{2/j}}\le\frac M{n\sigma^2}\le2$ in this case. (Observe that $\sigma^2\le1$, since $\sup |f(x_1,\dots,x_k)|\le1$.) Beside this, $\binom kj \frac{j!}{k!} \supp_{V\:V\subset \{1,\dots,k\},\,|V|=j} \(E(n^{-j}\bar I_{n,j}(f_V))^M\)^{1/M}= \frac{f_{\emptyset}}{k!}\le\frac{\sigma^2}{k!}$ in the case $j=0$. These estimates yield that $$ E(n^{-k}I_{n,k}(f^2))^M \le\sigma^{2M}\(\sum_{j=0}^k \frac{2^j}{(k-j)!} C_j^{1/2}\)^M $$ if $M\le 2n\sigma^2$, and we choose $C_0\ge1$. By formula (10.14) and this estimate $$ \aligned E(n^{-k/2}\bar I_{n,k}(f))^{2M}&\le 2^{3kM}M^{kM} E\(n^{-k}\bar I_{n,k}(f^2)\)^M \\ &\le\(\sum_{j=0}^k \frac{2^{3k+j}}{(k-j)!} C_j^{1/2}\)^M M^{kM}\sigma^{2M} \endaligned \tag10.17 $$ if $M\le 2n\sigma^2$. I show that with an appropriate choice of the coefficients $C_k$ (which may depend only on $k$ but not on $M$) the above estimate implies the inductive step. Indeed, we can choose such a sequence $C_k$, $=0,1,2,\dots$, with $C_0=1$ which satisfies the inequalities $C_k\ge\frac1{k!}$ and $$ \sum_{j=0}^k \frac{2^{3k+j}}{(k-j)!} C_j^{1/2}\le C_k\quad \text{ for all } k=1,2,\dots \tag10.18 $$ Let us choose such a sequence $C_k$, $k=0,1,\dots$, which satisfies these relations. Then formula $(10.1')$ holds for $M=1$, and our inductive procedure together with relations (10.17) and (10.18) imply that it also holds for $M\le 2n\sigma^2$, i.e. $$ E(n^{-k/2}I_{n,k}(f))^{2M} \le C_k^M\sigma^{2M}M^{kM}\quad\text {if }M=2^m\text{ and } M\le2n\sigma^2. $$ Thus we have proved Proposition~$10.1'$ in the special case when $M=2^m$, $m=0,1,\dots$, and $M\le 2n\sigma^2$. To estimate the moment $E|n^{-k/2}\bar I_{n,k}(f)|^{2M}$ for a general exponent $1\le M\le n\sigma^2$ (the number $M$ may be non-integer) let us consider the number $\bar M=\bar M(M)$ of the form $\bar M=2^m$ with some integer $m$ which satisfies the relation $\bar M\le M<2\bar M$. By applying the already proved part of Proposition~$10.1'$ for $\bar M$ we can write $$ \align E|n^{-k/2}\bar I_{n,k}(f)|^{2M}&\le \(E|n^{-k/2}\bar I_{n,k}(f)|^{2\bar M}\)^{M/\bar M}\le \(C_k^{\bar M}\sigma^{2\bar M}\bar M^{k\bar M}\)^{M/\bar M}\\ &\le C_k^M\sigma^{2M} (2M)^{kM}=(2^kC_k)^M\sigma^{2M}M^{kM}. \endalign $$ Proposition $10.1'$ is proved. \medskip\noindent {\it The proof of Theorem $8.3'$.}\/ By the Markov inequality and Proposition~$10.1'$ $$ P\(|n^{-k/2}\bar I_{n,k}(f)|>u\)\le \frac{E|n^{-k/2}\bar I_{n,k}(f)|^{2M}}{u^{2M}}\le \(\frac{C_k\sigma^2 M^k}{u^2}\)^M $$ if $u>0$ and $1\le M\le n\sigma^2$. Let us choose $M=\frac1e\(\frac{u^2}{C_k\sigma^2}\)^{1/k}$. With this choice of the parameter~$M$ we get that $$ P\(|n^{-k/2}I_{n,k}(f)|>u\)\le e^{-kM}=\exp\left\{- \frac keC_k^{-1/k}\(\frac u\sigma\)^{2/k}\right\} \tag10.19 $$ if $\sqrt{C_k}e^{k/2}\sigma\le u\le e^{k/2}\sqrt{C_k}n^{k/2} \sigma^{k+1}$. Relation (10.19) implies formula (10.9). Indeed, formula (10.9) remains valid for $\sqrt{C_k}e^{k/2}n^{k/2}\sigma^{k+1}\le u\le n^{k/2}\sigma^{k+1}$ if the constant $kC_k^{-1/k}e^{-1}$ in the exponent at the right-hand side is replaced by $\alpha=\min(kC_k^{-1/k}e^{-1},k)$, (here we exploit that $P\(|n^{-k/2}I_{n,k}(f)|>u\)\le P\(|n^{-k/2}I_{n,k}(f)|>\sqrt{C_k}e^{k/2}n^{k/2}\sigma^{k+1}\)$ if $u\ge \sqrt{C_k}e^{k/2}n^{k/2}\sigma^{k+1}$), and it holds also for $0\le u\le \sqrt{C_k}e^{k/2}\sigma$ if the right-hand side is multiplied with a sufficiently large constant~$C$. \medskip As we have mentioned, Theorems $8.3'$ and Theorem 10.4 together imply Theorem~8.3. \beginsection 11. Some useful basic results This section contains the proof of Borell's inequality and Proposition~10.3 which can be considered as the multivariate version of the Marcinkiewicz--Zygmund inequality, more precisely of its more important part. \medskip\noindent {\script 11 a.) The proof of Borell's inequality formulated in Theorem~10.2.} \medskip\noindent Borell's inequality will be proved as the consequence of the following hypercontractive inequality for Rademacher functions. \medskip\noindent {\bf Theorem 11.1. The hypercontractive inequality for Rademacher functions.} {\it Let us consider two copies $(X,\Cal X,\mu)$ and $(Y,\Cal Y,\nu)=(X,\Cal X,\mu)$ of the measure space $(X,\Cal X,\mu)$, where $X=\{-1,1\}$, $\Cal X$ contains all subsets of $X$, and $\mu(\{1\})=\mu(\{-1\})=\frac12$. Given a real number $\gamma>0$ let us introduce the linear operator $\bold T_\gamma$ which maps the real (or complex) valued functions on the space $X$ to the real (or complex) valued functions on the space $Y$ which is defined by the relations $\bold T_\gamma r_0=r_0$, and $\bold T_\gamma r_1= \gamma r_1$, where $r_0(1)=r_0(-1)=1$, and $r_1(1)=1$, $r_1(-1)=-1$. For all $n=1,2,\dots$ let us consider the $n$-fold product $(X^n, \Cal X^n,\mu^n)$ and $(Y^n,\Cal Y^n,\nu^n)$ of the spaces $(X,\Cal X,\mu)$ and $(Y,\Cal Y,\nu)$ together with the $n$-fold product of the operator $\bold T^n_\gamma$ of the operator $\bold T_\gamma$ acting between these product spaces, (i.e. $\bold T^n_\gamma$ is the linear transformation for which $\bold T^n_\gamma (f_1(x_1)\cdots f_n(x_n))=\bold T_\gamma f_1(x_1)\cdots \bold T_\gamma f_n(x_n)$ for all products of the functions $f_s$, $1\le s\le n$, on the space $(X,\Cal X,\mu)$). The transformation $\bold T^n_\gamma$ from the space $L_q(X^n,\Cal X^n,\mu^n)$ to the space $L_p(Y^n,\Cal Y^n,\nu^n)$ has norm 1 for all $n=1,2,\dots$ if $10$, and $E|Z|^p=\|f\|_p^p$, $E|Z|^q=\|f\|_q^q$. Fix some numbers $11$, and $t\ge0$. We want to prove that $$ \[\int |U_tf(x)|^{p(t,q)}\mu(\,dx)\]^{1/p(t,q)}\le \[\int |f(x)|^q\mu(\,dx)\]^{1/q} \quad \text {for all }t\ge0\tag11.6 $$ and functions $f$ on $X$. (The general theory helps to find the `right' definition of the function $p(t,q)$. It is defined as the solution of the differential equation $\frac p{2(p-1)} \frac{dp(t)}{dt}=p$, $p(0)=q$. The coefficient $\frac p{2(p-1)}$ in this equation agrees with the coefficient appearing in the logarithmic Sobolev inequality (11.5).) Let us prove inequality (11.6) first for such functions $f(x)=a+br_1(x)$ for which $a$ and $b$ are real numbers and $a\ge|b|$. Given a function $f(x)=a+br_1(x)$ with $a\ge|b|$ define the function $F(t)=\[\int (U_tf(x))^{p(t,q)}\mu(\,dx)\]^{1/p(t,q)}$. Observe that $U_tf(x)=a+be^{-t}r_1(x)$, and $a\ge|b|e^{-t}$. Hence to prove (11.6) it is enough to show that $$ \frac{d\|U_t(f)\|_{p(t,q)}}{dt}=\frac{d F(t)}{dt}\le0 \quad \text{for all }t>0 \tag11.7 $$ which means that the function $F(t)$ is monotone decreasing, and in the proof we can apply the logarithmic Sobolev inequality for the functions $f_t(x)=U_tf(x)$. We have $$ \align \frac{dF(t)}{dt}&=F(t)\biggl[-\frac{p'(t,q)}{p(t,q)}\ln F(t) +\frac{p'(t,q)}{p(t,q)}\frac {\int U_tf(x)^{p(t,q)} \ln U_t f(x)\mu(\,dx)} {\int U_tf(x)^{p(t,q)}\mu(\,dx)} \\ &\qquad +\frac{\int U_tf(x)^{p(t,q)-1}(U_tf(x))'\mu(d\,x)} {\int U_tf(x)^{p(t,q)}\mu(\,dx)}\biggr], \endalign $$ where $G(t,\cdot)'$ means partial derivative with respect to the variable~$t$. Since $F(t)=\|U_t(f)\|_{p(t,q)}$, $\int U_tf(x)^{p(t,q)}\mu(\,dx)=\| U_t(f)\|_{p(t,q)}^{p(t,q)}$, $(U_tf(x))'=-BU_tf(x)$ by the definition of the operator $B$, $$ \int U_tf(x)^{p(t,q)-1}(U_tf(x))'\mu(d\,x)= -\int U_tf(x)^{p(t,q)-1}B(U_tf)(x)\mu(d\,x), $$ and $\frac{p(t,q)}{p'(t,q)}=\frac{p(t,q)}{2(p(t,q)-1)}$ with our choice of functions, the last formula implies that the inequality $\frac{dF(t)}{dt}\le0$ is equivalent to the relation $$ \align &-\|U_t(f)\|_{p(t,q)}^{p(t,q)}\ln\|U_t(f)\|+\int U_tf^{p(t,q)}(x)\ln U_tf(x)\mu(\,dx)\\ &\qquad-\frac p{2(p-1)}\int (U_tf)^{p(t,q)-1}(x)BU_tf(x)\mu((\,dx)\le0. \endalign $$ But this inequality follows from the logarithmic Sobolev inequality if it is applied for the function $U_t(f)$ with $\bar p=p(t,q)$. To prove relation (11.6) for a general function $f$ it is enough to check that $|U_t(f)|\le U_t(|f|)$, i.e. $|U_t(f)(1)|\le U_t(|f|)(1)$ and $|U_t(f)(-1)|\le U_t(|f|)(-1)$ for arbitrary function $f$ and $t\ge0$, since this relation has been already proved for the function $|f|$. But this relation simply follows from the following calculation. If $f(1)=A$, $f(-1)=B$, then $f(x)=\frac{A+B}2+\frac{A-B}2r_1(x)$, $U_tf(x)=\frac{A+B}2+e^{-t}\frac{A-B}2r_1(x)$, i.e. $U_tf(1)=\frac{1+e^{-t}}2A+\frac{1-e^{-t}}2B$, and $U_tf(-1)=\frac{1-e^{-t}}2A+\frac{1+e^{-t}}2B$, while $(U_t|f|)(\pm1)=\frac{1+e^{-t}}2|A|+\frac{1\mp e^{-t}}2|B|$. Let us fix some numbers $11$ let us apply inequality (11.5) in the already proven case $p=2$ for the function $f^{p/2}$. We get that $\frac p2\int f^p(x)\ln f(x)\mu(\,dx)\le \int f^{p/2}(x)Bf^{p/2}(x)\mu(dx)+\frac p2\|f\|_p^p\ln\|f\|_p$. Hence to prove Proposition~11.4 in the general case it is enough to show that $$ \int f^{p/2}(x)Bf^{p/2}(x)\mu(\,dx)\le\frac{p^2}{4(p-1)} \int f^{p-1}(x) Bf(x)\mu(\,dx) $$ for a function $f(x)=a+br_1(x)$ such that $a\ge|b|$. The expressions in the last inequality can be simply calculated. As $$ f^{p/2}(x)=\[\(\frac{a+b}2\)^{p/2}+\(\frac{a-b}2\)^{p/2}\] +\[\(\frac{a+b}2\)^{p/2}-\(\frac{a-b}2\)^{p/2}\]r_1(x), $$ and $$ f^{p-1}(x)=\[\(\frac{a+b}2\)^{p-1}+\(\frac{a-b}2\)^{p-1}\] +\[\(\frac{a+b}2\)^{p-1}-\(\frac{a-b}2\)^{p-1}\]r_1(x) $$ this inequality can be rewritten as $$ \align &\[\(\frac{a+b}2\)^{p/2}-\(\frac{a-b}2\)^{p/2}\]^2 \\ &\quad \le \frac{p^2}{4(p-1)} \[\(\frac{a+b}2\)^{p-1}-\(\frac{a-b}2\)^{p-1}\] \(\frac{a+b}2-\frac{a-b}2\) \endalign $$ or $$ \(\int_u^v t^{(p-2)/2}\,dt\)^2\le \int_u^v t^{p-2}\,dt\cdot \int_u^v 1 \,dt $$ with $u=\frac{a-|b|}2$ and $v=\frac{a+|b|}2$. But the last formula is a simple consequence of the Schwarz inequality. Proposition 11.4 is proved. \medskip\noindent {\it Remark:} Theorem 11.3 is sharp in the following sense. The transformation $T_\gamma$, $T_\gamma(a+br_1(x))=a+\gamma br_1(x)$ as a transformation from the $L_q(X,\Cal X,\mu)$ space to the space $L_p(X,\Cal X,\mu)$ with $1\sqrt{\frac{q-1}{p-1}}$. To see this let us compare the $L_q$ norm of $1+\delta r_1(x)$ with the $L_p$-norm of $T_\gamma r_1(x)=1+\gamma\delta r_1(x)$ for a small parameter $\delta>0$. We have $\|1+\delta r_1(x)\|_q=\[\frac12\((1+\delta)^q+(1-\delta)^q\)\]^{1/q} =\[1+\frac{q(q-1)}2\delta^2+O(\delta^3)\]^{1/q}=1+\frac{q-1}2\delta^2 +O(\delta^3)$. Similarly, $\|1+\gamma\delta r_1(x)\|_p=1+\frac{p-1}2 \gamma^2\delta^2+O(\delta^3)$, and these relations imply the above remark. \medskip\noindent {\script 11 b.) The proof of Proposition 10.3.} \medskip\noindent {\it Proof of Proposition 10.3.} Let us use the notation introduced in the formulation of Proposition~10.3, and take another $k$ independent copies $\bar\xi_1^{(j)}$,\dots, $\bar\xi_n^{(j)}$, $1\le j\le k$, of the random sequences $\xi_1,\dots,\xi_n$ which are also independent of the sequence $\e_1,\dots,\e_n$ appearing in the formulation of Proposition 10.3. Let $\Cal F$ denote the $\sigma$-algebra generated by the random variables $\xi^{(j)}_1,\dots,\xi^{(j)}_n$, $1\le j\le k$, and let us introduce the notation $\xi^{(j,1)}_l=\xi^{(j)}_l$, $\xi^{(j,-1)}_l=\bar\xi^{(j)}_l$, $1\le l\le n$ and $1\le j\le k$. Let $\Cal V_k$ denote the set of $\pm1$ sequences of length $k$, and for a $v\in\Cal V_k$ let $m(v)$ denote the number of the digits $-1$ in the sequence $v=(v(1),\dots,v(k))$. Observe that $E\(f\left.\(\xi_{l_1}^{(1,v(1))},\dots,\xi_{l_k}^{(k,v(k))}\)\right| \Cal F\)=0$ if the $\pm1$ sequence $(v(1),\dots,v(k))$ contains at least one coordinate $-1$, (this is the point of the proof where we exploit the canonical property of the function $f$), and $$ Ef\left.\(\xi_{l_1}^{(1,1)},\dots,\xi_{l_k}^{(k,1)}\right|\Cal F\)= f\(\xi_{l_1}^{(1)},\dots,\xi_{l_k}^{(k)}\)\quad\text{for all indices }1\le l_j\le n,\; 1\le j\le k. $$ These relations together with the Jensen-inequality for conditional expectations imply that $$ \align |\bar I_{n,k}(f)|^p&=\left|E\left.\(\frac1{k!}\sum_{v\in \Cal V_k} (-1)^{m(v)} \!\! \summ\Sb 1\le l_r\le n,\; r=1,\dots, k\\ l_r\neq l_{r'} \text{ if } r\neq r'\endSb \!\! f\(\xi_{l_1}^{(1,v(1))},\dots,\xi_{l_k}^{(k,v(k))}\)\right|\Cal F\)\right|^p\\ &\le E\(\left|\left.\frac1{k!}\sum_{v\in \Cal V_k} (-1)^{m(v)} \!\!\!\! \summ\Sb 1\le l_r\le n,\; r=1,\dots, k\\ l_r\neq l_{r'} \text{ if } r\neq r'\endSb \!\!\!\! f\(\xi_{l_1}^{(1,v(1))},\dots,\xi_{l_k}^{(k,v(k))}\)\right|^p \right|\Cal F\). \endalign $$ Hence $$ E|\bar I_{n,k}(f)|^p\le E\left|\frac1{k!}\sum_{v\in \Cal V_k} (-1)^{m(v)} \summ\Sb 1\le l_r\le n,\; r=1,\dots, k\\ l_r\neq l_{r'} \text{ if } r\neq r'\endSb f\(\xi_{l_1}^{(1,v(1))},\dots,\xi_{l_k}^{(k,v(k))}\)\right|^p. \tag11.8 $$ Let us introduce the random variables $$ \tilde I_{n,k}(f)=\frac1{k!}\sum_{v\in \Cal V_k} (-1)^{m(v)} \summ\Sb 1\le l_r\le n,\; r=1,\dots, k\\ l_r\neq l_{r'} \text{ if } r\neq r'\endSb f\(\xi_{l_1}^{(1,v(1))},\dots,\xi_{l_k}^{(k,v(k))}\) \tag11.9 $$ and $$ \tilde I_{n,k}(f,\e)=\frac1{k!}\sum_{v\in \Cal V_k} (-1)^{m(v)} \summ\Sb 1\le l_r\le n,\; r=1,\dots, k\\ l_r\neq l_{r'} \text{ if } r\neq r'\endSb \e_{l_1}\cdots\e_{l_k} f\(\xi_{l_1}^{(1,v(1))},\dots,\xi_{l_k}^{(k,v(k))}\). \tag$11.9'$ $$ Let us recall that the number $m(v)$ in these formula denotes the number of the digits $-1$ in the $\pm1$ sequence $v$ of length $k$, i.e. it counts how many random variables $\xi_{l_j}^{(j,1)}$, $1\le j\le k$, were replaced by the `secondary copy' $\xi_{l_j}^{(j,-1)}$ in the corresponding terms of the sum in (11.9) or $(11.9')$. I claim that the above defined two random variables $\tilde I_{n,k}(f)$ and $\tilde I_{n,k}(f,\e)$ have the same distribution. This statement will be formulated in a slightly more general form which will be useful in the further part of this work. \medskip\noindent {\bf Lemma 11.5.} {\it Let us consider a (non-empty) class of functions $\Cal F$ of $k$ variables $f(x_1,\dots,x_k)$ on the space $(X^k,\Cal X^k)$ together with the random variables $\tilde I_{n,k}(f)$ and $\tilde I_{n,k}(f,\e)$ defined in formulas (11.9) and $(11.9')$ for all $f\in \Cal F$. The joint distributions of the set of random variables $\{\tilde I_{n,k}(f);\, f\in\Cal F\}$ and $\{\tilde I_{n,k}(f,\e);\, f\in \Cal F\}$ agree.} \medskip\noindent {\it The proof of Lemma 11.5.}\/ We even claim that fixing an arbitrary sequence $u=(u(1),\dots,u(n))$, $u(l)=\pm1$, $1\le l\le n$, of length~$n$, the conditional distribution of the field $\{\tilde I_{n,k}(f,\e);\,f\in \Cal F\}$ under the condition that $(\e_1,\dots,\e_n)=u=(u(1),\dots,u(n))$ agrees with the distribution of the field of $\{\tilde I_{n,k}(f);\, f\in\Cal F\}$. Indeed, the random variables $\tilde I_{n,k}(f)$, $f\in\Cal F$, defined in (11.9) are functions of a random vector consisting of coordinates $(\xi_l^{(j)},\bar\xi_l^{(j)})=(\xi^{(j,1)}_l,\xi^{(j,-1)}_l)$, $1\le l\le n$, $1\le j\le k$, and the distribution of this random vector does not change if we replace the coordinates $(\xi_{l}^{(j)},\bar\xi_l^{(j)})=(\xi^{(j,1)}_l,\xi^{(j,-1)}_l)$, by $(\bar\xi_l^{(j)},\xi_l^{(j)})=(\xi^{(j,-1)}_l,\xi^{(j,1)}_l)$, for those indices $(j,l)$ for which $u(l)=-1$ (independently of the value of the parameter $j$) and do not modify these random vectors for those coordinates $(l,j)$ for which $u(l)=1$. Replacing the original vector in the definition of the expression $\tilde I_{n,k}(f)$ in (11.9) for all $f\in \Cal F$ by this modified vector we carry out a measure preserving transformation. On the other hand, the random field we get in such a way has the same distribution as the conditional distribution of the random field $\tilde I_{n,k}(f,\e)$, $f\in\Cal F$, with the elements defined in $(11.9')$ under the condition that $(\e_1,\dots,\e_n)=u$ with $u=(u(1),\dots,u(n))$. To prove the last statement let us observe that the conditional distribution of the random field $\tilde I_{n,k}(f,\e)$, $f\in\Cal F$, under the condition $(\e_1,\dots,\e_n)=u$ is the same as that of the random field we obtain by putting $u_l=\e_l$, $1\le l\le n$, in all coordinates $\e_l$ of the random variables $\tilde I_{n,k}(f,\e)$. On the other hand, the random variables we get in such a way agree with the random variables we get by carrying out the above described transformation for the random variables $\tilde I_{n,k}(f)$, only the terms in the sums defining these random variables are listed in a different order. \medskip Relation (11.8) and the agreement of the distribution of the random variables $\tilde I_{n,k}(f)$ in (11.9) and $\tilde I_{n,k}(f)$ $(11.9')$ imply that $$ E|\bar I_{n,k}(f)|^p \le E\left|\frac1{k!}\sum_{v\in \Cal V_k} (-1)^{m(v)} \!\!\!\! \summ\Sb 1\le l_j\le n,\; j=1,\dots,k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \!\!\!\! \e_{l_1}\cdots\e_{l_k} f\(\xi_{l_1}^{(1,v(1))},\dots,\xi_{l_k}^{(k,v(k))}\)\right|^p. \tag11.10 $$ Let us define for all $v=(v(1),\dots,v(k))\in\Cal V_k$ the random variable $$ \bar I_{n,k,v}(f,\e)=\frac1{k!}\summ\Sb 1\le l_j\le n,\;j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \e_{l_1}\cdots\e_{l_k} f\(\xi_{l_1}^{(1,v(1))},\dots,\xi_{l_k}^{(k,v(k))}\), \quad v\in V_k. $$ The distribution of the random variables $\bar I_{n,k,v}(f,\e)$ agree with that of $\bar I_{n,k}(f,\e)$ introduced in (10.6) for all $v\in \Cal V_k$. Hence relation (11.10) implies that $$ \align E|\bar I_{n,k}(f)|^p &\le E\left|\sum_{v\in \Cal V_k} (-1)^{m(v)}\bar I_{n,k,v}(\e,f) \right|^p\\ &\le 2^{(k-1)p}\sum_{v\in\Cal V_k} E|\bar I_{n,k,v}(f,\e)|^p =2^{kp}E|\bar I_{n,k}(f,\e)|^p. \endalign $$ Proposition 10.3 is proved. \beginsection 12. Reduction of the main result in this work The main result of this paper is Theorem 8.4 or its multiple integral version Theorem~8.2. It can be considered as the multivariate version of Theorem 4.1, and its proof is also based on a similar argument. Following the method of the proof of Theorem~4.1 first we prove a multivariate version of Proposition~6.1 in Proposition~12.1 and reduce Theorem~8.4 to a simpler statement formulated in Proposition~12.2. The hard part of the problem is the proof of Proposition~12.2. In the first step of its proof we reduce it with the help of Theorem 10.4 (proved by de la Pe\~na and Montgomery--Smith) to an analogous result formulated in Proposition~$12.2'$, where the $U$-statistics to be investigated are replaced by their decoupled $U$-statistics counterpart introduced in Section~10. The proof of this result is simpler, because here we have more independence. It is based on a symmetrization argument, similar to the proof of Proposition~6.2. The details of this symmetrization argument will be explained in the next section. This section contains only an important preliminary result needed in this argument, a multi-dimensional variant of Hoeff\-ding's inequality (Theorem~ 3.4) formulated in Theorem~12.3. It yields an estimate about the distribution of homogeneous polynomials of Rademacher functions. The first result of this Section, Proposition~12.1, can be proved in almost the same way as its simplified version Proposition~6.1. The only essential difference between their proof is that Bernstein's inequality applied in the proof of Proposition 6.1 is replaced now by its multivariate version Theorem~8.3. Theorem~12.1 can be considered as the result we can get by means of the Theorem~8.3 and the chaining argument. Its main content, formulated in relation~(12.1) states that given a nice class of functions $\Cal F$ it has a subclass $\Cal F_{\bar\sigma}$ of relatively small cardinality which is also a relatively dense subclass of $\Cal F$ in the $L_2$ norm, and the supremum of the $U$-statistics with kernel functions from $\Cal F_{\bar\sigma}$ can be well bounded. To get an applicable result we also need some estimates on the number $\bar\sigma$ which measures how dense the subclass $\Cal F_{\bar\sigma}$ in $\Cal F$ is. Such estimates are contained at the end of this Proposition. In the formulation of Proposition~12.1 we introduce, similarly to Proposition~6.1, two parameters $\bar A>2^k$ and $M=M(\Bar A,k)$, and this may seem at first sight unnatural. But the introduction of these parameters turned out to be useful, they help, similarly to the analogous problem in Section~6 to fit the parameters in Propositions 12.1 and~12.2 as we want to apply them simultaneously. \medskip\noindent {\bf Proposition 12.1.} {\it Let us have the $k$-fold power $(X^k,\Cal X^k)$ of a measurable space $(X,\Cal X)$ with some probability measure $\mu$ on $(X,\Cal X)$ and a countable $L_2$-dense class $\Cal F$ of functions $f(x_1,\dots,x_k)$ of $k$ variables on $(X^k,\Cal X^k)$ with parameter $D$ and exponent~$L$, $L\ge1$, such that all functions $f\in\Cal F$ are canonical with respect to the measure $\mu$, and they satisfy conditions (8.4) and (8.5) with some real number $0<\sigma\le1$. Take a sequence of independent $\mu$-distributed random variables $\xi_1,\dots,\xi_n$, $n\ge\max(k,2)$, and consider the (degenerate) $U$-statistics $I_{n,k}(f)$, $f\in \Cal F$, defined in formula (8.7). Let us fix some number $\bar A\ge2^k$. For all numbers $M=M(k,\bar A)$ which are chosen sufficiently large in dependence of $\bar A$ and~$k$ the following relation depending on the numbers $\bar A$ and $M$ holds: For all numbers $u>0$ for which $n\sigma^2\ge\(\frac u\sigma\)^{2/k}\ge ML\log\frac2\sigma$ a number $\bar\sigma=\bar\sigma(u)$, $0\le\bar\sigma\le \sigma\le1$, and a collection of functions $\Cal F_{\bar\sigma}=\{f_1,\dots,f_m\} \subset\Cal F$ with $m\le D\bar\sigma^{-L}$ elements can be chosen in such a way that the sets $\Cal D_j=\{f\:f\in \Cal F, \int|f-f_j|^2\,d\mu \le\bar\sigma^2\}$, $1\le j\le m$, satisfy the relation $\bigcupp_{j=1}^m\Cal D_j=\Cal F$, and the (degenerate) $U$-statistics $I_{n,k}(f)$, $f\in\Cal F_{\bar\sigma(u)}$, satisfy the inequality $$ \aligned P&\(\sup_{f\in\Cal F_{\bar\sigma(u)}}n^{-k/2}|I_{n,k}(f)|\ge \frac u{\bar A}\)\le 2CD\exp\left\{-\alpha\(\frac u{10\bar A\sigma}\)^{2/k}\right\} \\ &\qquad \qquad \text{if}\quad n\sigma^2\ge \(\frac u\sigma\)^{2/k} \ge ML\log\frac2\sigma \endaligned \tag12.1 $$ with the constants $\alpha=\alpha(k)$, $C=C(k)$ appearing in formula~(8.9) of Theorem~8.3 and the exponent $L$ and parameter $D$ of the $L_2$-dense class $\Cal F$. The inequalities $4\(\frac u{\bar A\bar\sigma}\)^{2/k}\ge n\bar\sigma^2\ge\frac1{64}\(\frac u{\bar A\sigma}\)^{2/k}$ and $n\bar\sigma^2\ge\frac{M^{2/3}(L+\beta)\log n}{1000\bar A^{4/3}}$ also hold, provided that $n\sigma^2\ge \(\frac u\sigma\)^{2/k}\ge M(L+\beta)^{3/2}\log\frac2\sigma$ with $\beta=\max\(\frac{\log D}n,0\)$.} \medskip\noindent {\it Proof of Proposition 12.1.} Let us list the elements of the countable set $\Cal F$ as $f_1,f_2,\dots$. For all $p=0,1,2,\dots$ let us choose, by exploiting the $L_2$-density property of the class $\Cal F$, a set $\Cal F_p=\{f_{a(p,1)},\dots,f_{a(p,m_p)}\}\subset \Cal F$ with $m_p\le D\,2^{2pL}\sigma^{-L}$ elements in such a way that $\inff_{1\le j\le m_p}\int (f-f_{a(p,j)})^2\,d\mu\le 2^{-4p}\sigma^2$ for all $f\in\Cal F$. For all indices $a(j,p)$, $p=1,2,\dots$, $1\le j\le m_p$, choose a predecessor $a(j',p-1)$, $j'=j'(j,p)$, $1\le j'\le m_{p-1}$, in such a way that the functions $f_{a(j,p)}$ and $f_{a(j',p-1)}$ satisfy the relation $\int|f_{a(j,p)}-f_{a(j',p-1)}|^2\,d\mu\le \sigma^2 2^{-4(p-1)}$. Then we have $\int\(\frac{f_{a(j,p)}-f_{a(j',p-1)}}2\)^2\,d\mu\le4\sigma^2 2^{-4p}$ and $\supp_{x_j\in X,\,1\le j\le k}\left| \frac{f_{a(j,p)}(x_1,\dots,x_k)-f_{a(j',p-1)}(x_1,\dots,x_k)}2\right| \le 1$. Theorem~8.3 yields that $$ \aligned P(A(j,p))&=P\(n^{-k/2}|I_{n,k}(f_{a(j,p)}-f_{a(j',p-1)})|\ge \frac{2^{-(1+p)}u}{\bar A}\)\\ &\le C \exp\left\{-\alpha\(\frac{2^{p}u}{8\bar A \sigma}\)^{2/k} \right\} \quad \text {if}\quad 4n\sigma^2 2^{-4p}\ge\(\frac{2^{p}u} {8\bar A\sigma}\)^{2/k}, \\ &\qquad\qquad 1\le j\le m_p,\; p=1,2,\dots, \endaligned \tag12.2 $$ and $$ \align P(B(s))&=P\(n^{-k/2}|I_{n,k}(f_{0,s})|\ge \frac u{2\bar A}\)\le C\exp\left\{-\alpha\(\frac u{2\bar A\sigma}\)^{2/k}\right\}, \quad 1\le s\le m, \\ &\qquad\qquad\qquad\text{if} \quad n\sigma^2\ge \(\frac u{2\bar A\sigma}\)^{2/k}. \tag12.3 \endalign $$ Introduce an integer $R=R(u)$, $R>0$, which satisfies the relations $$ 2^{(4+{2/k})(R+1)}\(\frac{u}{\bar A\sigma}\)^{2/k} \ge 2^{2+6/k}n\sigma^2\ge 2^{(4+2/k)R}\(\frac{u}{\bar A\sigma}\)^{2/k}, $$ and define $\bar\sigma^2=2^{-4R}\sigma^2$ and $\Cal F_{\bar\sigma}=\Cal F_R$ (i.e the class of functions $\Cal F_p$ introduced before with $p=R$). (As $n\sigma^2\ge\(\frac u\sigma\)^{2/k}$ and $\bar A\ge2^k$ by our conditions, there exists such a positive integer $R$.) Then the cardinality~$m$ of the set $\Cal F_{\bar\sigma}$ is clearly not greater than $D\bar\sigma^{-L}$, and $\bigcupp_{j=1}^m \Cal D_j=\Cal F$. Beside this, the number $R$ was chosen in such a way that the inequalities (12.2) and (12.3) hold for $1\le p\le R$. Hence the definition of the predecessor of an index $a(j,p)$ implies that $$ \align &P\(\sup_{f\in\Cal F_{\bar\sigma}}n^{-k/2}|I_{n,k}(f)|\ge \frac u{\bar A}\) \le P\(\bigcup_{p=1}^R\bigcup_{j=1}^{m_p}A(j,p) \cup\bigcup_{s=1}^mB(s)\) \\ &\qquad \le \sum_{p=1}^R\sum_{j=1}^{m_p}P(A(j,p))+\sum_{s=1}^mP(B(s)) \le \sum_{p=1}^{\infty} CD\,2^{2pL}\sigma^{-L} \exp\left\{-\alpha\(\frac{2^{p}u}{8\bar A\sigma}\)^{2/k} \right\}\\ &\qquad\qquad +CD\sigma^{-L}\exp\left\{-\alpha\(\frac u{2\bar A\sigma}\)^{2/k}\right\}. \endalign $$ If the condition $\(\frac u\sigma\)^{2/k}\ge ML^{3/2} \log\frac2\sigma$ holds with a sufficiently large constant $M$ (depending on $\bar A$), then the inequalities $$ 2^{2pL}\sigma^{-L}\exp\left\{-\alpha\(\frac{2^{p}u}{8\bar A\sigma}\)^{2/k} \right\}\le 2^{-p} \exp\left\{-\alpha\(\frac{2^{p}u} {10\bar A \sigma}\)^{2/k} \right\} $$ hold for all $p=1,2,\dots$, and $$ \sigma^{-L}\exp\left\{-\alpha\(\frac u{2\bar A\sigma}\)^{2/k}\right\} \le\exp\left\{-\alpha\(\frac u{10\bar A\sigma}\)^{2/k}\right\}. $$ Hence the previous estimate implies that $$ \align &P\(\sup_{f\in\Cal F_{\bar\sigma}}n^{-k/2}|I_{n,k}(f)|\ge \frac u{\bar A}\) \le\sum_{p=1}^{\infty}CD 2^{-p} \exp\left\{-\alpha\(\frac{2^{p}u}{10\bar A \sigma}\)^{2/k} \right\}\\ &\qquad +CD\exp\left\{-\alpha\(\frac u{10\bar A \sigma}\)^{2/k}\right\} \le 2CD \exp\left\{-\alpha \(\frac u{10 \bar A\sigma}\)^{2/k}\right\}, \endalign $$ and relation (12.1) holds. We have $$ \align n\bar\sigma^2&=2^{-4R} n\sigma^2\le 2^{-4R}\cdot2^{(4+2/k)(R+1)-2-6/k}\(\frac{u}{\bar A\sigma}\)^{2/k}= 2^{2-4/k}\cdot 2^{2R/k}\(\frac{u}{\bar A \sigma}\)^{2/k}\\ &=2^{2-4/k}\cdot \(\frac\sigma{\bar\sigma}\)^{1/k}\(\frac{u}{\bar A \sigma}\)^{2/k}=2^{2-4/k}\cdot \(\frac{\bar\sigma}\sigma\)^{1/k} \(\frac{u}{\bar A \bar\sigma}\)^{2/k}, \endalign $$ hence $n\bar\sigma^2\le4\(\frac{u}{\bar A\bar\sigma}\)^{2/k}$. Beside this, as $n\sigma^2\ge2^{(4+2/k)R-2-6/k}\(\frac{u} {\bar A\sigma}\)^{2/k}$, $R\ge1$, $$ n\bar\sigma^2=2^{-4R}n\sigma^2\ge 2^{-2-6/k}\cdot 2^{2R/k}\(\frac u{\bar A\sigma}\)^{2/k}\ge\frac1{64}\(\frac u{\bar A\sigma}\)^{2/k}. $$ It remained to show that $n\bar\sigma^2\ge \frac{M^{2/3}(L+\beta)\log n}{1000\bar A^{4/3}}$. This inequality clearly holds under the conditions of Proposition~12.1 if $\sigma\le n^{-1/3}$, since in this case $\log\frac2\sigma\ge \frac{\log n}3$, and $n\bar\sigma^2\ge\frac1{64}\(\frac u {\bar A\sigma}\)^{2/k} \ge\frac1{64}\bar A^{-2/k} M(L+\beta)^{3/2}\log\frac2\sigma\ge \frac1{192}\bar A^{-2/k} M(L+\beta)\log n\ge \frac{M^{2/3}(L+\beta)\log n}{1000\bar A^{4/3}}$ if $M= M(\bar A,k)$ is chosen sufficiently large. If $\sigma\ge n^{-1/3}$, then the inequality $2^{(4+2/k)R}\(\frac u{\bar A\sigma}\)^{2/k} \le2^{2+6/k} n\sigma^2$ holds. Hence $2^{-4R}\ge 2^{-4(2+6/k))/(4+2/k)} \[\dfrac{\(\frac u{\bar A\sigma}\)^{2/k}}{n\sigma^2}\]^{4/(4+2/k)}$, and $$ n\bar\sigma^2=2^{-4R}n\sigma^2\ge\frac{2^{-16/3}}{\bar A^{4/3}} (n\sigma^2)^{1-\gamma}\[\(\frac u\sigma\)^{2/k}\]^\gamma \text{ with } \gamma=\frac4{4+\frac2k}\ge\frac23. $$ Since $n\sigma^2\ge(\frac u\sigma)^{2/k}\ge\frac M3(L+\beta)^{3/2}$, and $n\sigma^2\ge n^{1/3}$, the above estimates yield that $(n\sigma^2)^{1-\gamma}\[\(\frac u\sigma\)^{2/k}\]^\gamma\ge (n\sigma^2)^{1/3}\[\(\frac u\sigma\)^{2/k}\]^{2/3}$, and $n\bar\sigma^2\ge \frac{\bar A^{-4/3}}{50} (n\sigma^2)^{1/3}\[\(\frac u\sigma\)^{2/k}\]^{2/3}\ge \frac{\bar A^{-4/3}}{50}n^{1/9}\(\frac M3\)^{2/3} (L+\beta) \ge\frac{M^{2/3}(L+\beta)\log n}{1000 \bar A^{4/3}}$. \medskip Now we formulate a multivariate analog of Proposition~6.2 in Proposition~12.2 and show that Propositions~12.1 and~12.2 imply Theorem~8.4. \medskip\noindent {\bf Proposition 12.2.} {\it Let us have a probability measure $\mu$ on a measurable space $(X,\Cal X)$ together with a sequence of independent and $\mu$ distributed random variables $\xi_1,\dots,\xi_n$ and a countable $L_2$-dense class $\Cal F$ of canonical kernel functions $f=f(x_1,\dots,x_k)$ (with respect to the measure~$\mu$) with some parameter $D$ and exponent $L$ on the product space $(X^k,\Cal X^k)$ such that all functions $f\in\Cal F$ satisfy conditions (8.4) and (8.5) with some $0<\sigma\le1$, and consider the (degenerate) $U$-statistics $I_{n,k}(f)$ with the random sequence $\xi_1,\dots,\xi_n$ and kernel functions $f\in\Cal F$. There exists a sufficiently large constant $K=K(k)$ together with some numbers $\bar C=\bar C(k)>0$, $\gamma=\gamma(k)>0$ and threshold index $A_0=A_0(k)>0$ depending only on the order $k$ of the $U$-statistics such that if $n\sigma^2>K(L+\beta)\log n$ with $\beta=\max\(\frac{\log D}{\log n},0\)$, then the degenerate $U$-statistics $I_{n,k}(f)$, $f\in\Cal F$, satisfy the inequality $$ P\(\sup_{f\in\Cal F}|n^{-k/2}I_{n,k}(f)|\ge A n^{k/2}\sigma^{k+1}\) \le \bar C e^{-\gamma A^{1/2k}n\sigma^2}\quad \text{if } A\ge A_0. \tag12.4 $$ } \medskip We shall prove formula (8.10) by applying Proposition~12.2 with the choice $\sigma=\bar\sigma=\bar\sigma(u)$ defined in Proposition~12.1 and the classes $\Cal F=\Cal D_j$, more precisely the classes $\Cal F=\left\{\frac{g-f_j}2\: g\in\Cal D_j\right\}$ of functions introduced also in Proposition~12.1, where $f_j$ is the function appearing in the definition of the class of functions $\Cal D_j$. Clearly, $$ \aligned &P\(\supp_{f\in\Cal F}n^{-k/2}|I_{n,k}(f)|\ge u\)\le P\(\sup_{f\in\Cal F_{\bar\sigma}}n^{-k/2}|I_{n,k}(f)|\ge \frac u{\bar A}\) \\ &\qquad\qquad +\sum_{j=1}^m P\(\sup_{g\in\Cal D_j} n^{-k/2} \left|I_{n,k}\(\frac{f_j-g}2\)\right| \ge\(\frac12-\frac1{2\bar A}\)u\), \endaligned \tag12.5 $$ where $m$ is the cardinality of the set of functions $\Cal F_{\bar\sigma}$ appearing in Proposition~12.1. We want to show that if first $\bar A$ and then $M\ge M_0(\bar A)$ are chosen sufficiently large in Proposition~12.1, then the second term at the right-hand side of formula~(12.5) can be well bounded by means of Proposition 12.2, and Theorem~8.4 can be proved by means of this estimate. To carry out this program let us choose a number $\bar A_0$ in such a way that $\bar A_0\ge A_0$ and $\gamma\bar A_0^{1/2k}\ge\frac1K$ with the numbers $A_0$, $K$ and $\gamma$ in Proposition~12.2, put $\bar A=\max(2^{k+2}\bar A_0,2^k)$, and apply Proposition 12.1 with this number~$\bar A$. Then by Proposition~12.1 and the choice of the numbers $\bar A$ and $\bar A_0$ also the inequality $\(\frac u{\bar\sigma}\) ^{2/k}\ge\frac{\bar A^{2/k}}4n \bar\sigma^2\ge(4\bar A_0)^{2/k}n\bar\sigma^2$ holds, hence $u\ge 4\bar A_0 n^{k/2}\bar\sigma^{k+1}$ with the number $\bar\sigma$ in Proposition~12.1. This implies that $\(\frac12-\frac1{2\bar A}\)u\ge\frac u4\ge\bar A_0 n^{k/2}\bar\sigma^{k+1}$, $\bar A_0\ge A_0$, and by replacing the expression $\(\frac12-\frac1{2\bar A}\)u$ by $\bar A_0 n^{k/2}\bar\sigma^{k+1}$ in the probabilities of the sum in the second term at the right-hand side of (12.5) we enlarge them. The numbers $u$ considered in these estimations satisfy the condition $n\sigma^{2/k}\ge \(\frac u\sigma\)^{2/k} \ge M(L+\beta)^{3/2}\log\frac2\sigma$ imposed in Proposition~12.1 with some appropriately chosen constant $M$. Choose the number $M\ge M(\bar A,k)$ in Proposition~12.1 (which also can be chosen as the number~$M$ in formula~(8.10) of Theorem~8.4) in such a way that it also satisfies the inequality $\frac{M^{2/3}(L+\beta)\log n}{1000\bar A^{4/3}}\ge K(L+\beta)\log n$ with the number $K$ appearing in the conditions of Proposition~12.2. With such a choice the inequality $n\bar\sigma^2 \ge\frac{M^{2/3}(L+\beta)\log n}{1000\bar A^{4/3}} \ge K(L+\beta)\log n$ holds, and Proposition 12.2 can be applied to bound the terms in the sum at the right-hand side of (12.5). It yields the estimate $$ \align &P\(\sup_{g\in\Cal D_j} n^{-k/2} \left|I_{n,k}\(\frac{f_j-g}2\)\right| \ge\(\frac12-\frac1{2\bar A}\)u\)\\ &\qquad\le P\(\sup_{g\in\Cal D_j}n^{-k/2} \left|I_{n,k}\(\frac{f_j-g}2\)\right| \ge\bar A_0n^{k/2}\bar\sigma^{k+1}\) \le \bar Ce^{-\gamma\bar A_0^{1/2k}n\bar\sigma^2} \endalign $$ for all $1\le j\le m$. (Observe that the set of functions $\frac{f_j-g}2,\;g\in\Cal D_j$ is an $L_2$-dense class with parameter $D$ and exponent $L$.) Hence Proposition~12.1 (relation (12.1) together with the inequality $m\le D\bar \sigma^{-L}$) and formula 12.4 with $A=\bar A_0$ imply that $$ P\(\supp_{f\in\Cal F} n^{-k/2}|I_{n,k}(f)|\ge u\) \le 2CD\exp\left\{-\alpha\(\frac u{10\bar A\sigma}\)^{2/k}\right\} +\bar CD\bar\sigma^{-L} e^{-\gamma\bar A_0^{1/2k}n\bar\sigma^2}. \tag12.6 $$ To get the result of Theorem~8.4 from inequality (12.6) we have to replace its second term at the right-hand side with a more appropriate expression where, in particular, we get rid of the coefficient $\bar\sigma^{-L}$. The condition $n\bar\sigma^2\ge K(L+\beta)\log n$ implies that $\bar\sigma\ge n^{-1/2}$, and by our choice of $\bar A_0$ we have $\gamma \bar A_0^{1/2k}n\bar\sigma^2\ge \frac1Kn\bar\sigma^2 \ge L\log n\ge 2L\log\frac1{\bar \sigma}$, i.e. $\bar\sigma^{-L}\le e^{\gamma\bar A_0^{1/2k}n\bar\sigma^2/2}$. By the estimates of Proposition~12.1 $n\bar\sigma^2 \ge\frac1{64}\(\frac u{\bar A\sigma}\)^{2/k}$. The above relations imply that $\bar\sigma^{-L} e^{-\gamma\bar A_0^{1/2k}n \bar\sigma^2}\le e^{-\gamma\bar A_0^{1/2k}n\bar\sigma^2/2}\le \exp\left\{-\frac\gamma{128} \bar A_0^{1/2k} \bar A^{-2/k}\(\frac u\sigma\)^{2/k}\right\}$. Hence relation (12.6) yields that $$ \align &P\(\supp_{f\in\Cal F}n^{-k/2}|I_{n,k}(f)|\ge u\)\\ &\qquad\le 2CD\exp \left\{-\frac\alpha{(10\bar A)^2}\(\frac u\sigma\)^{2/k}\right\} +\bar CD\exp\left\{-\frac\gamma{128} \bar A_0^{1/2k} \bar A^{-2/k} \(\frac u\sigma\)^{2/k}\right\}, \endalign $$ and this estimate implies Theorem~8.4. \medskip Thus to complete the proof of Theorem~8.4 it is enough to prove Proposition~12.2. It turned out to be useful to apply an approach similar to the proof of Theorem~8.3. In the proof of Theorem~8.3 first an appropriate counterpart of this result was proved, where the $U$-statistics were replaced by their decoupled $U$-statistics analogs defined in formula (10.5), and then the desired result was deduced from this estimate and Theorem~10.4. Similarly, Proposition 12.2 will be deduced from the following result. \medskip\noindent {\bf Proposition $12.2'$.} {\it Let a class of functions $f\in\Cal F$ on the $k$-fold product $(X^k,\Cal X^k)$ of a measurable space $(X,\Cal X)$, a probability measure $\mu$ on $(X,\Cal X)$ together with a sequence of independent and $\mu$ distributed random variables $\xi_1,\dots,\xi_n$ satisfy the conditions of Proposition~12.2. Let us take $k$ independent copies $\xi_1^{(j)},\dots,\xi_n^{(j)}$, $1\le j\le k$, of the random sequence $\xi_1,\dots,\xi_n$, and consider the decoupled $U$-statistics $\bar I_{n,k}(f)$, $f\in \Cal F$, defined with their help by formula (10.5). Then there exists a sufficiently large constant $K=K(k)$ together with some number $\gamma=\gamma(k)>0$ and threshold index $A_0=A_0(k)>0$ depending only on the order $k$ of the decoupled $U$-statistics we consider such that if $n\sigma^2>K(L+\beta)\log n$ with $\beta=\max\(\frac{\log D}{\log n},0\)$, then the (degenerate) decoupled $U$-statistics $\bar I_{n,k}(f)$, $f\in\Cal F$, satisfy the following version of inequality (12.4): $$ P\(\sup_{f\in\Cal F}|n^{-k/2}\bar I_{n,k}(f)|\ge A n^{k/2}\sigma^{k+1}\) \le e^{-\gamma A^{1/2k}n\sigma^2}\quad \text{if } A\ge A_0 .\tag12.7 $$ }\medskip It is clear that Proposition~$12.2'$ and Theorem~10.4, more explicitly formula~$(10.8')$ in it imply Proposition 12.2. The proof of Proposition~12.2 is based on a symmetrization argument which will be explained in the next section. Here an important ingredient of it will be proved, the multivariate version of Hoeff\-ding's inequality formulated in Theorem~3.4. \medskip\noindent {\bf Theorem 12.3. The multivariate version of Hoeffding's inequality.} {\it Let $\e_1$,\dots, $\e_n$ be independent random variables, $P(\e_j=1)=P(\e_j=-1)=\frac12$, $1\le j\le n$. Fix a positive integer~$k$, and define the random variable $$ Z=\sum\Sb (j_1,\dots, j_k)\: 1\le j_l\le n \text{ for all } 1\le l\le k\\ j_l\neq j_{l'} \text{ if }l\neq l' \endSb a(j_1,\dots, j_k) \e_{j_1}\cdots \e_{j_k} \tag12.8 $$ with the help of some real numbers $a(j_1,\dots,j_k)$ which are given for all sets of indices such that $1\le j_l\le n$, $1\le l\le k$, and $j_l\neq j_{l'}$ if $l\neq l'$. Put $$ S^2=\sum\Sb (j_1,\dots, j_k)\: 1\le j_l\le n \text{ for all } 1\le l\le k\\ j_l\neq j_{l'} \text{ if }l\neq l' \endSb a^2(j_1,\dots, j_k) \tag12.9 $$ Then $$ P(|Z|>u)\le C \exp\left\{-B\(\frac uS\)^{2/k}\right\} \quad\text{for all }u\ge 0 \tag12.10 $$ with some constants $B>0$ and $C>0$ depending only on the parameter $k$. Relation (12.10) holds for instance with the choice $B=\frac k{2e(k!)^{1/k}}$ and $C=e^k$.} \medskip\noindent {\it Proof of Theorem 12.3.}\/ We get with the help of formula (10.4) (which is a consequence of Borell's inequality) that $$ E|Z|^q\le(q-1)^{kq/2} \(EZ^2\)^{q/2} \le q^{kq/2} \(EZ^2\)^{q/2}= q^{kq/2} \bar S^q $$ with $$ \bar S^2=\sum_{1\le j_1u)\le \(q^{k/2}\frac {\sqrt{k!}S}u\)^q \quad \text{for all } u>0\quad \text{and } q\ge2. $$ Choose the number $q$ as the solution of the equation $q\(\frac {\sqrt{k!}S}u\)^{2/k}=\frac1e$. Then we get that $P(|Z|>u)\le \exp\left\{- B\(\frac uS\)^{2/k}\right\}$ with $B=\frac k{2e(k!)^{1/k}}$, provided that $q=\frac1{e{k!}^{1/k}} \(\frac uS\)^{2/k}\ge2$, i.e. $B\(\frac uS\)^{2/k}\ge k$. By multiplying the above upper bound with $C=e^k$ we get such an estimate for $P(|Z|>u)$ which holds for all $u>0$. \medskip\noindent {\it Remark:}\/ The result of Theorem~12.3 will be good enough for our purposes, although the constants $B$ and $C$ we have given in formula (12.10) are not optimal. Thus Theorem~3.4 yields that in the special case $k=1$ the estimate (12.10) holds with $B=\frac12$ and $C=1$ (and not only with $B=\frac1{2e}$ and $C=e$). The reason for this relative weakness of Theorem~12.3 is that the moment estimate given for a homogeneous polynomial of Rademacher functions in formula (10.4) is not always sharp. In Theorem~16.6 I present (without proof) an improved version of Theorem~12.3 which yields the estimate (12.10) with the right constant~$C$ in the exponent. The proof can be found in paper~[22]. It is based on a sharp estimate on the moments $EZ^{2M}$ for large positive integers~$M$ formulated in Theorem~16.7. This estimate can be considered as the improvement of Bernstein's inequality in a most important special case. \beginsection 13. The strategy of the proof for the main result of this paper We have reduced the proof of the main result of this paper, the proof of Theorem~8.4 to that of Proposition~$12.2'$. It is a multivariate version of Proposition~6.2, and also its proof is based on similar ideas. In particular, a multivariate version of Lemma~7.2 will be proved, which means some kind of randomization. In this result we consider a class of decoupled, degenerate $U$-statistics $\bar I_{n,k}(f)$ together with a class of randomized, decoupled $U$-statistics $\bar I_{n,k}(f,\e)$ defined in formulas (10.5) and (10.6) respectively with the same countable class of functions $f\in\Cal F$ and want to bound the probability $P\(n^{-k/2}\supp_{f\in\Cal F}\bar I_{n,k}(f)|>u\)$ with the help of a probability of the form $P\(n^{-k/2}\supp_{f\in\Cal F}\bar I_{n,k}(f,\e)|>Bu\)$ with some appropriate universal constant $B>0$. To carry out such a program we introduce $2k$ independent copies $\xi_1^{(j)},\dots,\xi_n^{(j)}$ and $\bar\xi_1^{(j)},\dots,\bar\xi_n^{(j)}$, $1\le j\le n$, of the sequence of random variables $\xi_1,\dots,\xi_n$ we have at the start. We shall work with these $2k$ copies and a sequence of independent random variables $\e=(\e_1,\dots,\e_n)$, $P(\e_l=1)=P(\e_l=-1)=\frac12$, $1\le l\le n$, independent also of the sequences $\xi_1^{(j)},\dots,\xi_n^{(j)}$ and $\bar\xi_1^{(j)},\dots,\bar\xi_n^{(j)}$, $1\le j\le k$. Given some function $f(x_1,\dots,x_k)$ of $k$ variables in the $k$-fold power $(X^k,\Cal X^k)$ of some measurable space $(X,\Cal X)$ let us consider the random sums $\tilde I_{n,k}(f)$ and $\tilde I_{n,}(f,\e)$ defined in formulas (11.9) and $(11.9')$ with the help of the above random sequences. We shall use Lemma~11.5 which states that given a class of functions of $k$-variables $f\in \Cal F$, the joint distribution of the random variables $\tilde I_{n,k}(f)$ and $\tilde I_{n,}(f,\e)$ agree. As we shall see later, Lemma~11.5 enables us to reduce the multivariate version of Lemma~7.2 we would like to prove to an appropriate bounding of the distribution of $\supp_{f\in\Cal F}\bar I_{n,k}(f)$ by that of $\supp_{f\in\Cal F}\tilde I_{n,k}(f)$. In the proof of Lemma~7.2 we met a simple special case of this problem, and it could be solved by means of the Symmetrization Lemma (Lemma~7.1). In the general case Lemma~7.1 is not sufficient for our purposes, since we have to work with not necessarily independent random variables. Hence we prove a generalized version of it. \medskip\noindent {\bf Lemma 13.1 (Generalized version of the Symmetrization Lemma.)} {\it Let $Z_n$ and $\bar Z_n$, $n=1,2,\dots$, be two sequences of random variables on a probability space $(\Omega,\Cal A,P)$. Let a $\sigma$-algebra $\Cal B\subset \Cal A$ be given on the probability space $(\Omega,\Cal A,P)$ together with a $\Cal B$-measurable set $B$ and two numbers $\alpha>0$ and $\beta>0$ such that the random variables $Z_n$, $n=1,2,\dots$, are $\Cal B$ measurable, and the inequality $$ P(|\bar Z_n|\le\alpha|\Cal B)(\oo)\ge\beta\quad \text{for all } n=1,2,\dots \text{ if } \oo\in B \tag13.1 $$ holds. Then $$ P\(\sup_{1\le n<\infty}|Z_n|>\alpha+u\)\le\frac1\beta P\(\supp_{1\le n<\infty}|Z_n-\bar Z_n|>u\)+(1-P(B))\quad\text{for all } u>0. \tag13.2 $$ }\medskip\noindent {\it Proof of Lemma 13.1.}\/ Put $\tau=\min\{n\: |Z_n|>\alpha+u)$ if there exists such an $n$, and $\tau=0$ otherwise. Then $$ \align P(\{\tau=n\}\cap B)&\le\int_{\{\tau=n\}\cap B}\frac1\beta P(|\bar Z_n|\le \alpha|\Cal B)\,dP =\frac1\beta P(\{\tau=n\}\cap\{|\bar Z_n|\le\alpha\}\cap B)\\ &\le \frac1\beta P(\{\tau=n\}\cap\{|Z_n-\bar Z_n|>u\}) \quad \text{for all } n=1,2,\dots. \endalign $$ Hence $$ \align &P\(\sup_{1\le n<\infty}|Z_n|>\alpha+u\)-(1-P(B))\le P\(\left\{\sup_{1\le n<\infty}|Z_n|>\alpha+u\right\}\cap B\) \\ &\qquad=\sum_{n=1}^\infty P(\{\tau=n\}\cap B) \le \frac1\beta \sum_{n=1}^\infty P(\{\tau=n\}\cap\{|Z_n-\bar Z_n|>u\}) \\ &\qquad \le\frac1\beta P\(\supp_{1\le n<\infty}|Z_n-\bar Z_n|>u\). \endalign $$ Thus Lemma~13.1 is proved. \medskip The main difficulty we meet when we try to prove Proposition~$12.2'$ instead of its simpler version, Proposition~6.2 is that now we have to check an estimate of the form (13.1) with some appropriately chosen random variables $Z_n$, $\sigma$-algebra $\Cal B$ and set~$B$ instead of the estimate (7.1) applied in the proof of Proposition~6.2. The cause of this difference is that now we have to work with not completely independent random variables. In the symmetrization argument needed in the proof of Proposition~6.2 we could simply check inequality~(7.1) by calculating the variance of the random variables we were working with. On the other hand, to check inequality (13.1) in the symmetrization argument we want to apply in the present case we shall bound the conditional variance of certain random variables, and we can only state that this conditional variance is relatively small with great probability. In the proof of Proposition~$12.2'$ we formulate and prove a multivariate version of the definition of {\it good tail behaviour for a class of normalized random sums},\/ where the normalized random sums are replaced by degenerate decoupled $U$-statistics. It is enough to prove the good tail-behaviour of decoupled $U$-statistics introduced below by means of an appropriate induction, and Proposition~$12.2'$ follows from it. But to carry out such a program we have to formulate and check another property which will be called the {\it good tail behaviour for a class of integrals of decoupled $U$-statistics}. This property helps us to carry out the induction procedure needed in the proof of Proposition~$12.2'$. Its introduction and proof corresponds to the symmetrization argument formulated in Lemma~7.2 in the proof of Proposition~6.2. The above mentioned two properties will be proved simultaneously. Before their formulation I make some comments about the idea behind the introduction of the property `good tail behaviour for a class of integrals of decoupled $U$-statistics'. In the introduction of this property we consider a class of functions $f(x_1,\dots,x_k,y)$ depending on a parameter $y\in Y$, but in all further applications we shall apply this property with the choice $Y=X^l$ and $\rho=\mu^l$ (i.e. with the $l$-th power of the space $X$ and the probability measure $\mu$ on it) with some integer $l$. The property `good tail behaviour for a class of integrals of decoupled $U$-statistics' with the above choice will be useful for us for the following reason. We shall consider the expression introduced in formula (11.9), which is some sort of linear combination of decoupled $U$-statistics, and want to bound the inner sums at the right-hand side of this expression. More explicitly, we consider those inner sum terms for which $m(v)=l\ge1$, i.e.\ for which the original sample elements are replaced by their independent copies in $l\ge1$ coordinates. We want to calculate the conditional variance of such sums under the condition that the values of the elements of the original sample are prescribed. The property of good tail behaviour for a class of integrals of decoupled $U$-statistics helps us in getting a good estimate for these expressions. If we want to bound the conditional variance of such an inner sum where the original sample elements are replaced in $l$ coordinates, then the application of the property of good tail behaviour for a class of integrals of $U$-statistics with $k-l$ instead of $k$ parameters and with the choice $(Y,\Cal Y,\rho)=(X^l,\Cal X^l,\mu^l)$ will be useful. By applying this property with such a choice together with the canonical property of the function $f(x_1,\dots,x_k)$ we shall work with we can prove the estimate we need. Let me also remark that the estimate (13.5) we have imposed in the definition of the property of `good tail behaviour for a class of integrals of $U$-statistics' is fairly natural. We have applied the natural normalization, and with such a normalization it is natural to expect that the distribution of $\supp_{f\in\Cal F}n^{-k}H_{n,k}(f)$ behaves similarly to that of $\const\(\sigma\eta^2\)^k$, where $\eta$ is a standard normal random variables. Formula (13.5) expresses such a behaviour, only the power of the number~$A$ in the exponent at the right-hand side was chosen in a non-optimal way. Naturally, we want to prove the property of good tail behaviour for a class of integrals of decoupled $U$-statistics under appropriate, not too restrictive conditions. Let me remark that in the conditions of Proposition~13.3 we want to prove we have imposed beside formula (13.6) a fairly weak condition (13.7). Most difficulties arise in the proof because we want to work with this condition. Here we did not demand that the $L_2$-norm of the functions $f(x_1,\dots,x_k,y)$ should be small for all parameters~$y$. We only assumed that some average of these $L_2$-norms expressed in formula (13.7) are small. Now I formulate the definition of the properties we shall work with. \medskip\noindent {\bf Definition of good tail behaviour for a class of decoupled $U$-statistics.} {\it Let us have some measurable space $(X,\Cal X)$ and a probability measure $\mu$ on it. Let us consider some class $\Cal F$ of functions $f(x_1,\dots,x_k)$ on the $k$-fold product $(X^k,\Cal X^k)$ of the space $(X,\Cal X)$. Fix some positive integer~$n\ge k$ and positive number $0<\sigma\le1$, and take $k$ independent copies $\xi_1^{(j)},\dots,\xi_n^{(j)}$, $1\le j\le k$, of a sequence of independent $\mu$-distributed random variables $\xi_1,\dots,\xi_n$. Let us introduce with the help of these random variables the decoupled $U$-statistics $\bar I_{n,k}(f)$, $f\in\Cal F$, defined in formula~(10.5). Given some real number $T>0$ we say that the set of decoupled $U$-statistics determined by the class of functions $\Cal F$ has a good tail behaviour at level~$T$ (with parameters $n$ and $\sigma^2$ which we fix in the sequel) if the inequality holds: $$ P\(\sup_{f\in\Cal F}|n^{-k/2}\bar I_{n,k}(f)|\ge A n^{k/2}\sigma^{k+1}\) \le \exp\left\{-A^{1/2k}n\sigma^2 \right\} \quad \text{for all } A>T. \tag13.3 $$ }\medskip We shall also introduce the following property: \medskip\noindent {\bf Definition of good tail behaviour for a class of integrals of decoupled $U$-statistics.} {\it Let us have a product space $(X^k\times Y,\Cal X^k\times\Cal Y)$ with some product measure $\mu^k\times\rho$, where $(X^k,\Cal X^k,\mu^k)$ is the $k$-fold product of some probability space $(X,\Cal X,\mu)$, and $(Y,\Cal Y,\rho)$ is some other probability space. Fix some positive integer~$n\ge k$ and positive number $\sigma>0$, and consider some class $\Cal F$ of functions $f(x_1,\dots,x_k,y)$ on the product space $(X^k\times Y,\Cal X^k\times\Cal Y,\mu^k\times\rho)$. Take $k$ independent copies $\xi_1^{(j)},\dots,\xi_n^{(j)}$, $1\le j\le k$, of a sequence of independent, $\mu$-distributed random variables $\xi_1,\dots,\xi_n$. For all $f\in\Cal F$ and $y\in Y$ let us define the decoupled $U$-statistics $\bar I_{n,k}(f,y)=\bar I_{n,k}(f_y)$ by means of these random variables $\xi_1^{(j)},\dots,\xi_n^{(j)}$, $1\le j\le k$, the kernel function $f_y(x_1,\dots,x_k)=f(x_1,\dots,x_k,y)$ and formula~(10.5). Define with the help of these $U$-statistics $\bar I_{n,k}(f,y)$ the random integrals $$ H_{n,k}(f)=\int \bar I_{n,k}(f,y)^2\rho(\,dy), \quad f\in\Cal F. \tag13.4 $$ Choose some real number $T>0$. We say that the set of random integrals $H_{n,k}(f)$, $f\in\Cal F$, have a good tail behaviour at level $T$ (with parameters $n$ and $\sigma^2$ which we fix in the sequel) if $$ P\(\sup_{f\in\Cal F} n^{-k}H_{n,k}(f)\ge A^2 n^k\sigma^{2k+2}\) \le \exp\left\{-A^{1/(2k+1)}n\sigma^2 \right\} \quad \text{for } A> T. \tag13.5 $$ } Now I formulate those two inductive statements in Propositions~13.2 and~13.3 which imply that the above introduced properties of good tail behaviour for a class of decoupled $U$-statistics and good tail behaviour for a class of integrals of decoupled $U$-statistics hold under fairly general conditions. Proposition~$12.2'$ can be obtained as a relatively simple consequence of these results. \medskip\noindent {\bf Proposition 13.2.} {\it Let us fix a positive integer~$n\ge k$, a real number $0<\sigma\le2^{-(k+1)}$ and a probability measure $\mu$ on a measurable space $(X,\Cal X)$ together with a countable $L_2$-dense class $\Cal F$ of canonical kernel functions $f=f(x_1,\dots,x_k)$ (with respect to the measure~$\mu$) on the $k$-fold product space $(X^k,\Cal X^k)$ which has exponent $L\ge1$ and parameter~$D$. Let us also assume that all functions $f\in\Cal F$ satisfy the conditions $\supp_{x_j\in X, 1\le j\le k} |f(x_1,\dots,x_k)|\le 2^{-(k+1)}$, $\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)\le \sigma^2$, and $n\sigma^2>K(L+\beta)\log n$ with an appropriately chosen fixed number $K=K(k)$ with $\beta=\max\(\frac{\log D}{\log n},0\)$. There exists some real number $A_0=A_0(k)>1$ such that if all classes of functions $\Cal F$ satisfying the above conditions the sets of decoupled $U$-statistics determined $\bar I_{n,k}(f)$, $f\in\Cal F$, have a good tail behaviour at level~$T^{4/3}$ for some $T\ge A_0$, then they also have a good tail behaviour at level~$T$.} \medskip\noindent {\bf Proposition 13.3.} {\it Fix some positive integer $n\ge k$ and real number $0<\sigma\le2^{-(k+1)}$, and let us have a product space $(X^k\times Y,\Cal X^k\times\Cal Y)$ with some product measure $\mu^k\times\rho$, where $(X^k,\Cal X^k,\mu^k)$ is the $k$-fold product of some probability space $(X,\Cal X,\mu)$, and $(Y,\Cal Y,\rho)$ is some other probability space. Let us have a countable $L_2$-dense class $\Cal F$ of canonical functions $f(x_1,\dots,x_k,y)$ on the product space $(X^k\times Y,\Cal X^k\times\Cal Y,\mu^k\times\rho)$ with some exponent $L\ge1$ and parameter $D$. Let us also assume that the functions $f\in \Cal F$ satisfy the conditions $$ \supp_{x_j\in X, 1\le j\le k, y\in Y}|f(x_1,\dots,x_k,y)|\le 2^{-(k+1)} \tag13.6 $$ and $$ \int f^2(x_1,\dots,x_k,y)\mu(\,dx_1)\dots\mu(\,dx_k)\rho(\,dy) \le\sigma^2 \quad \text{for all } f\in \Cal F. \tag13.7 $$ Let the inequality $n\sigma^2>K(L+\beta)\log n$ hold with a sufficiently large, appropriately chosen number $K=K(k)$ and $\beta=\max\(\frac{\log D}{\log n},0\)$. There exists some number $A_0=A_0(k)>1$ such that if for all classes of functions $\Cal F$ which satisfy the above conditions the random integrals $H_{n,k}(f)$, $f\in\Cal F$, defined in (13.4) have a good tail behaviour at level $T^{(2k+1)/2k}$ with some $T\ge A_0$, then they also have a good tail behaviour at level~$T$.} \medskip\noindent {\it Remark:}\/ In the conditions of Proposition~13.3 the notion of canonical functions appeared in a slightly more general form than it was defined in formula~(8.8). We say that a function $f(x_1,\dots,x_k,y)$ on the product space $(X^k\times Y,\Cal X^k\times\Cal Y,\mu^k\times\rho)$ is canonical if $$ \align &\int f(x_1,\dots,x_{j-1},u,x_{j+1},\dots,x_k,y)\mu(\,du)=0\\ &\qquad \qquad \text{for all } 1\le j\le k,\; x_s\in X, \;s\neq j \text{ and }y\in Y \endalign $$ and $$ \int f(x_1,\dots,x_k,y)\rho(\,dy)=0\quad \text{for all } x_j\in X, \; 1\le j\le k. $$ \medskip It is not difficult to deduce Proposition~$12.2'$ from Proposition~13.2. Indeed, let us observe that the set of decoupled $U$-statistics determined by a class of functions $\Cal F$ satisfying the conditions of Proposition~13.2 has a good tail-behaviour at level $T_0=\sigma^{-(k+1)}$, since under the conditions of this Proposition the probability at the left-hand side of (13.3) equals zero for $A>\sigma^{-(k+1)}$. Then we get from Proposition~13.2 by induction with respect to the number $j$, that this set of decoupled $U$-statistics has a good tail-behaviour also for all $T\ge T_0^{(3/4)^j}=\sigma^{-(k+1)(3/4)^j}$ for $j=0,1,2,\dots$ if $\sigma^{-(k+1)(3/4)^j}\ge A_0$. (Observe that $\sigma<1$ under the conditions of Proposition~13.2, since $\sigma^2\le 2^{-2(k+1)}$ in this case.) This implies that if a class of functions $\Cal F$ satisfies the conditions of Proposition~13.2, then the set of decoupled $U$-statistics determined by this class of functions has a good tail-behaviour at level $T=A_0^{4/3}$, i.e. at a level which depends only on the order $k$ of the decoupled $U$-statistics. This result implies Proposition~$12.2'$, only we have to apply it not directly for the class of functions~$\Cal F$ appearing in it, but these functions have to be multiplied by a sufficiently small positive number depending only on~$k$. Similarly to the above argument an inductive procedure yields a corollary of Proposition~13.3 formulated below. Actually, we shall need this corollary of Proposition~13.3. \medskip\noindent {\bf Corollary of Proposition 13.3.} {\it If the class of functions $\Cal F$ satisfies the conditions of Proposition~13.3, then there exists a constant $\bar A_0=\bar A_0(k)>0$ depending only on $k$ such that the class of integrals $H_{n,k}(f)$, $f\in \Cal F$ defined in formula (13.4) have a good tail behaviour at level $\bar A_0$.} \medskip The main difficulty in the proof of Proposition 13.2 appears as we try to apply the symmetrization procedure corresponding to Lemma~7.2 in the one-variate case. This difficulty can be overcome by means of Proposition~13.3, more precisely its corollary. It helps us to estimate the conditional variances of the decoupled $U$-statistics we have to handle in the proof of Proposition~13.2. The proof of Propositions~13.2 and~13.3 apply similar arguments, and they will be proved simultaneously. These results will be proved by means of the following inductive procedure. First Propositions~13.2 and then Proposition~13.3 are proved for $k=1$. If Propositions~13.2 and~13.3 are already proved for all $k'0$ such that the inequality $$ \aligned P\(\sup_{f\in\Cal F} n^{-k/2}\left|\bar I_{n,k}(f)\right|>An^{k/2}\sigma^{k+1}\)&< 2^{k+1}P\(\sup_{f\in\Cal F} \left|\bar I_{n,k}^{\e}(f)\right| >2^{-(k+1)}A n^k\sigma^{k+1}\)\\ &\qquad+2^kn^{k-1}e^{-A^{1/(2k-1)} n\sigma^2/k} \endaligned \tag14.2 $$ holds for all $A\ge A_0$.} \medskip To formulate Lemma~14.1B first we have to introduce some new quantities. We introduce them, because we want to adapt the symmetrization argument of Lemma~11.5 to the case when we work with a function $f(x_1,\dots,x_k,y)$ depending on a parameter~$y$, and we have to introduce some new notions in this new situation. Some of the quantities introduced below will be used somewhat later. The quantities $\bar I_{n,k}^V(f,y)$ introduced in (14.3) will depend on the sets $V\subset\{1,\dots,k\}$, and they are the natural adaptations of the inner sum terms in formula (14.9). Such expressions are needed when we want to formulate that version of the symmetrization result of Lemma~11.5 which is needed in the proof of Proposition~13.3. Their randomizations $\bar I_{n,k}^{(V,\e)}(f,y)$, introduced in formula (14.6), correspond to the inner sum terms in formula $(11.9')$. We also introduce the integrals of these expressions in formulas~(14.4) and~(14.7). Let us consider a class $\Cal F$ of functions $f(x_1,\dots,x_k,y)\in \Cal F$ on a space $(X^k\times Y, \Cal X^k \times \Cal Y,\mu^k\times\rho)$ which satisfies the conditions of Proposition~13.3. Let us take $2k$ independent copies $\xi_1^{(j)},\dots,\xi_n^{(j)}$, $\bar\xi_1^{(j)},\dots,\bar\xi_n^{(j)}$, $1\le j\le k$, of a sequence of independent $\mu$ distributed random variables $\xi_1,\dots,\xi_k$ together with a sequence of independent random variables $(\e_1,\dots,\e_n)$, $P(\e_l=1)=P(\e_l=-1)=\frac12$, $1\le l\le n$, which is also independent of the previous random sequences. Let us introduce the notation $\xi_l^{(j,1)}=\xi_l^{(j)}$ and $\xi_l^{(j,-1)}=\bar\xi_l^{(j)}$, $1\le l\le n$, $1\le j\le k$. For all subsets $V\subset\{1,\dots,k\}$ of the set $\{1,\dots,k\}$ let $|V|$ denote the cardinality of this set, and define for all functions $f(x_1,\dots,x_k,y)\in \Cal F$ and $V\subset\{1,\dots,k\}$ the decoupled $U$-statistics $$ \bar I_{n,k}^V(f,y)=\frac1{k!}\summ\Sb 1\le l_j\le n,\;j=1,\dots,k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb f\(\xi_{l_1}^{(1,\delta_1)},\dots,\xi_{l_k}^{(k,\delta_k)},y\), \tag14.3 $$ where $\delta_j=\pm1$, $1\le j\le k$, $\delta_j=1$ if $j\in V$, and $\delta_j=-1$ if $j\notin V$, together with the random variables $$ H_{n,k}^V(f)=\int \bar I_{n,k}^V(f,y)^2\rho(\,dy), \quad f\in\Cal F. \tag14.4 $$ Put $$ \bar I_{n,k}(f,y)=\bar I_{n,k}^{\{1,\dots,k\}}(f,y),\quad H_{n,k}(f)=H_{n,k}^{\{1,\dots,k\}}(f), \tag14.5 $$ i.e. $\bar I_{n,k}(f,y)$ and $H_{n,k}(f)$ are the random variables $\bar I_{n,k}^V(f,y)$ and $H_{n,k}^V(f)$ with $V=\{1,\dots,k\}$ which means that these expressions are defined with the help of the random variables $\xi_l^{(j,1)}$, $1\le j\le k$, $1\le l\le n$. Let us also define the `randomized version' of the random variables $\bar I_{n,k}^V(f,y)$ and $H_{n,k}^V(f)$ as $$ \bar I_{n,k}^{(V,\e)}(f,y)=\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \e_{l_1}\cdots\e_{l_k}f\(\xi_{l_1}^{(1,\delta_1)},\dots, \xi_{l_k}^{(k,\delta_k)},y\),\quad f\in\Cal F, \tag14.6 $$ and $$ H_{n,k}^{(V,\e)}(f)=\int \bar I_{n,k}^{(V,\e)}(f,y)^2\rho(\,dy) ,\quad f\in\Cal F, \tag14.7 $$ where $\delta_j=1$ if $j\in V$, and $\delta_j=-1$ if $j\in\{1,\dots,k\}\setminus V$. Let us also introduce the random variables $$ \bar W(f)=\int\[\sum_{V\subset \{1,\dots,k\}} (-1)^{|V|}\bar I_{n,k}^{(V,\e)}(f,y)\]^2\rho(\,dy), \quad f\in\Cal F \tag14.8 $$ With the help of the above notations we can formulate Lemma~14.1B. \medskip\noindent {\bf Lemma 14.1B.} {\it Let $\Cal F$ be a set of functions on $(X^k\times Y,\Cal X^k\times\Cal Y)$ which satisfies the conditions of Proposition~13.3 with some probability measure $\mu^k\times\rho$. Let us have $2k$ independent copies $\xi_{1}^{j,\pm1},\dots,\xi_{n}^{j,\pm1}$, $1\le j\le k$, of a sequence of independent $\mu$ distributed random variables $\xi_1,\dots,\xi_n$ together with a sequence of independent random variables $\e_1,\dots,\e_n$, $P(\e_j=1)=P(\e_j=-1)=\frac12$, $1\le j\le n$, which is independent also of the previously considered sequences. Then there exists some constant $A_0=A_0(k)>0$ such that if the integrals $H_{n,k}(f)$, $f\in\Cal F$, determined by this class of functions $\Cal F$ have a good tail behaviour at level $T^{(2k+1)/2k}$ for some $T\ge A_0$, (this property was defined in Section~13 in the definition of good tail behaviour for a class of integrals of decoupled $U$-statistics), then the inequality $$ \aligned P\(\sup_{f\in\Cal F} \left|H_{n,k}(f)\right|>A^2n^{2k}\sigma^{2(k+1)}\) &<2P\(\sup_{f\in\Cal F} \left|\bar W(f)\right| >\frac{A^2}2 n^{2k}\sigma^{2(k+1)}\)\\ &\qquad+2^{2k+1}n^{k-1}e^{-A^{1/2k} n\sigma^2/k} \endaligned \tag 14.9 $$ holds with the random variables $H_{n,k}(f)$ introduced in the second identity of relation (14.5) and with $\bar W(f)$ defined in formula(14.8) for all $A\ge T$.} \medskip We formulate a corollary of Lemma~14.1B which can be better applied than the original lemma. The inconvenience in Lemma~14.B arises, because at the right-hand side of formula (14.9) we have a probability depending on $\supp_{f\in\Cal F} |\bar W(f)|$, and $\bar W(f)$ is a too complicated expression. It equals the integral of the square of homogeneous polynomials of Rademacher functions (with random coefficients) depending on a parameter $y$ with respect to this parameter. We have to understand better the structure of $\bar W(f)$. Hence we shall rewrite it by means of relations (14.10) and (14.11) in a somewhat complicated, but more explicit form. These formulas enable us to find such a corollary of Lemma~14.B which is more appropriate for us. To work out the details first we introduce some diagrams. Let $\Cal G=\Cal G(k)$ denote the set of all diagrams consisting of two rows, such that each row is the set $\{1,\dots,k\}$, and the diagrams of $\Cal G$ contain some edges $\{(j_1,j_1')\dots,(j_s,j_s')\}$, $0\le s\le k$, connecting some point (vertex) of the first row with some point (vertex) of the second row. The vertices $j_1,\dots,j_s$ which are end points of some edge in the first row are all different, and the same relation holds also for the vertices $j_1',\dots,j_s'$ in the second row. Given some diagram $G\in\Cal G$ let $e(G)=\{(j_1,j_1')\dots,(j_s,j_s')\}$ denote the set of its edges, and let $v_1(G)=\{j_1,\dots,j_s\}$ be the set of those vertices in the first row and $v_2(G)=\{j_1',\dots,j_s'\}$ the set of those vertices in the second row of the diagram~$G$ from which an edge of~$G$ starts. Given some diagram $G\in \Cal G$ and two sets $V_1,V_2\subset\{1,\dots,k\}$, we define with the help of the random variables $\xi_{1}^{(j,1)},\dots,\xi_{n}^{(j,1)}$, $\xi_{1}^{(j,-1)},\dots,\xi_{n}^{(j,-1)}$, $1\le j\le k$, and $\e=(\e_1,\dots,\e_n)$ taking part in the definition of the random variables $\bar W(f)$ the following random variables $H_{n,k}(f|G,V_1,V_2)$: $$ \aligned H_{n,k}(f|G,V_1,V_2)&=\sum\Sb l_1,\dots,l_k,\,l'_1,\dots,l'_k\\ 1\le l_j\le n,\, l_j\neq l_{j'} \text{ if }j\neq j',\,1\le j,j'\le k,\\ 1\le l'_j\le n,\, l'_j\neq l'_{j'}\text { if } j\neq j',\,1\le j,j'\le k,\\ l_j=l'_{j'} \text { if } (j,j')\in e(G),\; l_j\neq l'_{j'} \text { if } (j,j')\notin e(G)\endSb\!\!\!\!\!\!\!\!\!\!\!\! \prod_{j\in\{1,\dots,k\} \setminus v_1(G)} \!\!\!\! \e_{l_j} \prod_{j\in\{1,\dots,k\} \setminus v_2(G)} \!\!\!\! \e_{l'_j} \\ &\qquad\frac1{k!^2} \int f(\xi_{l_1}^{(1,\delta_1)},\dots,\xi_{l_k}^{(k,\delta_k)},y) f(\xi_{l'_1}^{(1,\bar\delta_1)},\dots,\xi_{l'_k}^{(k,\bar\delta_k)},y) \rho(\,dy) \endaligned \tag14.10 $$ where $\delta_j=1$ if $j\in V_1$, $\delta_j=-1$ if $j\notin V_1$, and $\bar\delta_j=1$ if $j\in V_2$, $\bar\delta_j=-1$ if $j\notin V_2$. (Let us observe that if the graph $G$ contains $s$ edges, then the product of the $\e$-s in (14.10) contains $2(k-s)$ terms and the number of terms in the sum (14.10) is of order $n^{2k-s}$.) As the Corollary of Proposition~14.1B will indicate in the proof of Proposition~13.3 the expression $H_{n,k}(f|G,V_1,V_2)$ has to be estimated. This can be done by means of Theorem 12.3, the multivariate version of Hoeffding's inequality. But the estimate we get in such a way has to be rewritten in a form appropriate for our inductive procedure. This will be done in the next section. We shall prove that the identity $$ \bar W(f)=\sum_{G\in \Cal G,\, V_1,V_2\subset\{1,\dots,k\}} (-1)^{|V_1|+|V_2|} H_{n,k}(f|G,V_1,V_2) \tag14.11 $$ holds. To prove this identity let us write first $$ \bar W(f)=\sum_{V_1,V_2\subset \{1,\dots,k\}} (-1)^{|V_1|+|V_2|} \int\bar I_{n,k}^{(V_1,\e)}(f,y)\bar I_{n,k}^{(V_2,\e)}(f,y)\rho(\,dy). $$ Then let us express the products $\bar I_{n,k}^{(V_1,\e)}(f,y)\bar I_{n,k}^{(V_2,\e)}(f,y)$ by means of formula (14.6). Let us rewrite this product as a sum of products of the form $\frac1{k!^2}\prodd_{j=1}^k\e_{l_j}f(\cdots) \prodd_{j=1}^k\e_{l_j'}f(\cdots)$ and let us define the following partition of the terms in this sum. The elements of this partition are indexed by the diagrams $G\in \Cal G$, and if we take a diagram $G\in\Cal G$ with the set of edges $e(G)= \{(j_1,j_1'),\dots,(j_s,j_s')\}$, then the term of this sum determined by the indices $l_1,\dots,l_k,l'_1,\dots,l'_k$ belongs to the element of the partition indexed by this diagram $G$ if and only if $l_{j_u}=l_{j_u'}'$ for all $1\le u\le s$, and no more numbers between the indices $l_1,\dots,l_k,l_1'\dots,l'_k$ may agree. Since $\e_{l_{j_u}}\e_{l_{j_u'}'}=1$ for all $1\le u\le s$ and all other $\e_{l_j}$ and $\e_{l_j'}$ are different for a term of the sum in the element of the partition indexed by the diagram~$G$ we get by integrating the product $\bar I_{n,k}^{(V_1,\e)}(f,y)\bar I_{n,k}^{(V_2,\e)}(f,y)$ with respect to the measure $\rho$ that $$ \int\bar I_{n,k}^{(V_1,\e)}(f,y)\bar I_{n,k}^{(V_2,\e)}(f,y)\rho(\,dy) =\sum_{G\in \Cal G} H_{n,k}(f|G,V_1,V_2) $$ for all $V_1,V_2\in\{1,\dots,k\}$. The last two relations imply formula (14.11). Since the number of terms in the sum of formula (14.11) is less than $2^{4k}k!$, this relation implies that Lemma~14.1B has the following corollary: \medskip\noindent {\bf Corollary of Lemma 14.1B.} {\it Let a set of functions $\Cal F$ satisfy the conditions of Proposition~13.3. Then there exists some constant $A_0=A_0(k)>0$ such that if the integrals $H_{n,k}(f)$, $f\in\Cal F$, determined by this class of functions $\Cal F$ have a good tail behaviour at level $T^{(2k+1)/2k}$ for some $T\ge A_0$, then the inequality $$ \aligned &P\(\sup_{f\in\Cal F} |H_{n,k}(f)|>A^2n^k\sigma^{2(k+1)}\)\\ &\qquad\qquad\le 2\sum_{G\in \Cal G,\, V_1,V_2\subset\{1,\dots,k\}} P\(\sup_{f\in\Cal F} \left |H_{n,k}(f|G,V_1,V_2)\right| >\frac{A^2}{2^{4k+1}k!} n^k\sigma^{2(k+1)}\) \\ &\qquad\qquad\qquad +2^{2k+1}n^{k-1}e^{-A^{1/2k} n\sigma^2/k} \endaligned \tag14.12 $$ holds with the random variables $H_{n,k}(f)$ and $H_{n,k}(f|G,V_1,V_2)$ defined in formulas (14.5) and (14.10) for all $A\ge T$.} \medskip\noindent In the proof of Lemmas 14.1A and 14.1B we apply the result of the following Lemmas~14.2A and~14.2B. \medskip\noindent {\bf Lemma 14.2A.} {\it Let us take $2k$ independent copies $$ \xi_1^{(j,1)},\dots,\xi_n^{(j,1)} \quad \text{and} \quad \xi_1^{(j,-1)},\dots,\xi_n^{(j,-1)}, \quad 1\le j\le k, $$ of a sequence of independent $\mu$ distributed random variables $\xi_1,\dots,\xi_k$ together with a sequence of independent random variables $(\e_1,\dots,\e_n)$, $P(\e_l=1)=P(\e_l=-1)=\frac12$, $1\le l\le n$, which is also independent of the previous sequences. Let $\Cal F$ be a class of functions which satisfies the conditions of Proposition 13.2. Introduce with the help of the above random variables for all sets $V\subset\{1,\dots,k\}$ and functions $f\in \Cal F$ the decoupled $U$-statistic $$ \bar I_{n,k}^V(f)=\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb f\(\xi_{l_1}^{(1,\delta_1)},\dots,\xi_{l_k}^{(k,\delta_k)}\) \tag14.13 $$ and its `randomized version' $$ \bar I_{n,k}^{(V,\e)}(f)=\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \e_{l_1}\cdots\e_{l_k}f\(\xi_{l_1}^{(1,\delta_1)},\dots, \xi_{l_k}^{(k,\delta_k)}\), \quad f\in\Cal F, \tag$14.13'$ $$ where $\delta_j=\pm1$, and $\delta_j=1$ if $j\in V$, and $\delta_j=-1$ if $j\in\{1,\dots,k\}\setminus V$. Then the sets of random variables $$ S(f)=\sum_{V\subset \{1,\dots,k\}} (-1)^{|V|}\bar I_{n,k}^V(f), \quad f\in\Cal F \tag14.14 $$ and $$ \bar S(f)=\sum_{V\subset \{1,\dots,k\}} (-1)^{|V|}\bar I_{n,k}^{(V,\e)}(f), \quad f\in\Cal F \tag$14.14'$ $$ have the same joint distribution.} \medskip\noindent {\bf Lemma 14.2B.} {\it Let us take $2k$ independent copies $$ \xi_1^{(j,1)},\dots,\xi_n^{(j,1)}\quad \text{and} \quad \xi_1^{(j,-1)},\dots,\xi_n^{(j,-1)}, \quad 1\le j\le k, $$ of a sequence of independent $\mu$ distributed random variables $\xi_1,\dots,\xi_k$ together with a sequence of independent random variables $(\e_1,\dots,\e_n)$, $P(\e_l=1)=P(\e_l=-1)=\frac12$, $1\le l\le n$, which is also independent of the previous sequences. Let $\Cal F$ be a class of functions of $k$ variables satisfying the conditions of Proposition 13.3. For all functions $f\in \Cal F$ and $V\in\{1,\dots,k\}$ consider the decoupled $U$-statistics $\bar I_{n,k}^V(f,y)$ defined by formula (14.3) with the help of the random variables $\xi_1^{(j,1)},\dots,\xi_n^{(j,1)}$ and $\xi_1^{(j,-1)},\dots,\xi_n^{(j,-1)}$, and define with their help the random variables $$ W(f)=\int\[\sum_{V\subset \{1,\dots,k\}} (-1)^{|V|}\bar I_{n,k}^V(f,y)\]^2\rho(\,dy), \quad f\in\Cal F. \tag14.15 $$ Then the random vectors $\{W(f)\: f\in \Cal F\}$ defined in (14.15) and $\{\bar W(f)\: f\in \Cal F\}$ defined in (14.8) have the same distribution.} \medskip\noindent {\it Proof of Lemmas 14.2A and 14.2B.} Lemma 14.2A agrees actually with the already proved result Lemma~11.5, only the notation is different. The proof of Lemma~14.2B is also similar to the proof of~11.5. We can state that even the following stronger statement holds. For any $\pm1$ sequence $(u_1,\dots,u_n)$ of length~$n$ the conditional distribution of the random field $\bar W(f)$, $f\in\Cal F$, under the condition $(\e_1,\dots,\e_n)=(u_1,\dots,u_n)$ agrees with the distribution of the random field $W(f)$, $f\in\Cal F$. To see this relation let us first observe that the conditional distribution of the field $\bar W(f)$ under this condition agrees with the distribution of the random field we get by replacing the random variables $\e_l$ by $u_l$ for all $1\le l\le n$ in formulas (14.6) and (14.8). Beside this, we get by replacing the vectors $(\xi^{(j,1)}_l,\xi^{(j,-1)}_l)$ by $(\xi^{(j,-1)}_l,\xi^{(j,1)}_l)$ for those indices $(j,l)$ for which $u(l)=-1$ (independently of the value of the parameter $j$) and not modifying these vectors with coordinates $(l,j)$ such that $u(l)=1$ a measure preserving transformation of the distribution of the random vector consisting of the random variables $(\xi^{(j,1)}_l,\xi^{(j,-1)}_l)$, $1\le l\le n$, $1\le j\le k$, and this implies that also the distribution of the field $W(f)$, $f\in\Cal F$, agrees with the distribution of the field we obtain by carrying out the above transformation in the elements of the field $W(f)$, $f\in\Cal F$. These facts imply Lemma~14.2B. \medskip Now we formulate and prove Lemma~14.3A. \medskip\noindent {\bf Lemma 14.3A.} {\it Let us consider a class of functions $\Cal F$ satisfying the conditions of Proposition 13.2 with parameter~$k$, and the random variables $\bar I_{n,k}^V(f)$, $f\in\Cal F$, $V\subset\{1,\dots,k\}$, defined in formula (14.1). Let $\Cal B=\Cal B(\xi_{1}^{(j,1)},\dots,\xi_{n}^{(j,1)};\;1\le j\le k)$ denote the $\sigma$-algebra generated by the random variables $\xi_{1}^{(j,1)},\dots,\xi_{n}^{(j,1)}$ , $1\le j\le k$, i.e.\ by the random sequences with second coordinate 1 in their upper index taking part in the definition of the random variables $\bar I_{n,k}^V(f)$. For all $V\in\{1,\dots,k\}$, $V\neq\{1,\dots,k\}$, there exists a number $A_0=A_0(k)>0$ such that the inequality $$ P\(\sup_{f\in\Cal F}\left.E\(\bar I_{n,k}^V(f)^2\right|\Cal B\) > 2^{-(3k+3)}A^2n^{2k}\sigma^{2k+2}\)< n^{k-1}e^{-A^{1/(2k-1)} n\sigma^2/k}. \tag14.16 $$ holds for all $A\ge A_0$.} \medskip\noindent {\it Proof of Lemma 14.3A.}\/ Let us first consider the case $V=\emptyset$. In this case we can write $\left.E\(\bar I_{n,k}^\emptyset(f)^2\right|\Cal B\)=E\(\bar I_{n,k}^\emptyset(f)^2\) \le\frac{n^k}{k!}\sigma^2\le n^{2k}\sigma^{2k+2}$ for all $f\in\Cal F$. In the above calculation we have exploited that the functions $f\in\Cal F$ are canonical, and this implies certain orthogonalities, and beside this the inequality $n\sigma^2\ge1$ holds. The above relation implies that for $V=\emptyset$ the probability at the left-hand side of (14.16) equals zero if the number $A_0$ is chosen sufficiently large, i.e. the inequality (14.16) holds in this case. To avoid some complications in the notation let us first restrict our attention to sets of the form $V=\{1,\dots,u\}$ with some $1\le u 2^{-(3k+3)}A^2n^{2k}\sigma^{2k+2} \right\}\\ &\qquad \subset \bigcup \Sb 1\le l_j\le n\; j=u+1,\dots,k\\ l_j\neq l_{j'} \text { if } j\neq j'\endSb \left\{\oo\: \sup_{f\in\Cal F}\left. E\(\bar I_{n,k}^V(f,l_{u+1},\dots,l_k)^2\right|\Cal B\)(\oo) >\frac{A^2n^{2k}\sigma^{2k+2}}{2^{(3k+3)}n^{k-u}} \right\}. \endaligned \tag14.18 $$ The probability of the events in the union at the right-hand side of (14.18) can be estimated with the help of the corollary of Proposition~13.3 with parameter $u\frac {A^2n^{k+u}\sigma^{2k+2}} {2^{(3k+3)}}\)\le e^{-A^{1/(2u+1)}(n-u)\sigma^2}. \tag14.19 $$ Indeed, introduce the space $(Y,\Cal Y,\rho)=(X^{k-u},\Cal X^{k-u},\mu^{k-u})$, the $k-u$-fold power of the measure space $(X, \Cal X,\mu)$, and for the sake of simpler notations write $y=(x_{u+1}, \dots,x_k)$ for a point $y\in Y$. Let us consider a class of functions $f\in \Cal F$ which satisfies the conditions of Proposition~13.2 and let us prove for it relation (14.16). Let us introduce for this goal the class of those function $\bar{\Cal F}$ on the space $(X^u\times Y,\Cal X^u\times\Cal Y,\mu^u\times\rho)$ which can be written in the form $\bar f(x_1,\dots,x_u,y) =f(x_1,\dots,x_k)$ with $y=(x_{u+1},\dots,x_k)$ and some function $f(x_1,\dots,x_k)\in\Cal F$. The class of functions $\bar{\Cal F}$ satisfies the conditions of Proposition~13.3 with parameter $uA_0(u), \endaligned \tag14.20 $$ where $$ H_{n-u,u}(\bar f)=\int I_{n,u}(\bar f,y)^2\rho(\,dy)=\frac{k!}{u!} E\(\bar I_{n,k}^V(f,l_{u+1},\dots,l_k)^2|\Cal B\) \tag14.21 $$ with the function $f\in\Cal F$ for which the identity $\bar f(x_1,\dots,x_u,y)=f(x_1,\dots,x_k)$ holds with $y=(x_{u+1},\dots,x_k)$. It is not difficult to deduce formula (14.19) from relations (14.20) and (14.21). It is enough to replace the level $\frac{A^2n^{k+u}\sigma^{2k+2}}{2^{(3k+3)}}$ in the probability at the left-hand side of (14.19) by $A^2(n-u)^{2u} \sigma^{2u+2} <\frac {A^2n^{k+u}\sigma^{2k+2}}{2^{(3k+3)}}$. The last inequality really holds if the constant $K$ in the condition $n\sigma^2>K\log n$ in Proposition~13.2 is chosen sufficiently large. Relations (14.18) and (14.19) imply that $$ P\(\sup_{f\in\Cal F}\left. E\(\bar I_{n,k}^V(f)^2\right| \Cal B\)(\oo) > 2^{-(3k+3)}A^2n^{2k}\sigma^{2k+2} \)\le n^{k-u}e^{-A^{1/(2u+1)}(n-u)\sigma^2}. $$ Since $e^{-A^{1/(2u+1)}(n-u)\sigma^2}\le e^{-A^{1/(2k-1)}n\sigma^2/k}$ if $u\le k-1$ and $n\ge k$ inequality (14.16) holds for a set $V$ of the form $V=\{1,\dots,u\}$, $1\le uAn^{k/2}\sigma^{k+1}\)&< 2P\(\sup_{f\in\Cal F} |S(f)|>\frac A2n^k\sigma^{k+1}\)\\ &\qquad +2^kn^{k-1}e^{-A^{1/(2k-1)} n\sigma^2/k} \endaligned \tag14.22 $$ with the function $S(f)$ defined in (14.14). To prove relation (14.22) introduce the random variables $Z(f)=I_{n,k}^{\{1,\dots,k\}}(f)$ and $\bar Z(f)=-\summ_{V\subset \{1,\dots,k\},\,V\neq\{1,\dots,k\}} (-1)^{|V|}\bar I_{n,k}^V(f)$ for all $f\in\Cal F$, the $\sigma$-algebra $\Cal B$ considered in Lemma~14.3A and the set $$ B=\bigcap\Sb V\subset\{1,\dots,k\}\\V\neq\{1,\dots,k\}\endSb \left\{\oo\: \sup_{f\in\Cal F}\left.E\(\bar I_{n,k}^V(f)^2\right| \Cal B\)(\oo) \le 2^{-(3k+3)}A^2n^{2k}\sigma^{2k+2}\right\}. $$ Observe that $S(f)=Z(f)-\bar Z(f)$, $f\in\Cal F$, $B\in\Cal B$, and by Lemma~14.3A the inequality $1-P(B)\le2^kn^{k-1}e^{-A^{1/(2k-1)} n\sigma^2/k}$ holds. Hence to prove relation (14.22) it is enough to apply Lemma~13.1 and to show that $$ P\(|\bar Z(f)|>\frac A2n^k\sigma^{k+1}|\Cal B\)(\oo)\le\frac12 \quad \text{ for all }f\in\Cal F \quad \text {if } \oo\in\Cal B. \tag14.23 $$ But $P\(\bar I_{n,k}^{|V|}(f)|>2^{-(k+1)} An^k\sigma^{k+1}|\Cal B\)(\oo)\le 2^{-(k+1)}$ for all functions $f\in \Cal F$ and sets $V\subset\{1,\dots,k\}$, $V\neq\{1,\dots,k\}$, if $\oo\in B$ by the `conditional Chebishev inequality', hence relations (14.23) and~(14.22) hold. Lemma 14.1A follows from relation (14.22), Lemma~14.2A and the observation that the random vectors $\{\bar I_{n,k}^{(V,\e)}(f)\}$, $f\in\Cal F$, defined in $(14.13')$ have the same distribution for all $V\in\{1,\dots,k\}$ as the random vector $\bar I_{n,k}^{\e}(f)$, defined in formula~(14.1). Hence $$ P\(\sup_{f\in\Cal F} |S(f)|>\frac A2n^k\sigma^{k+1}\)\le 2^kP\(\sup_{f\in\Cal F} \left|\bar I_{n,k}^\e(f)\right| >2^{-(k+1)}A n^k\sigma^{k+1}\). $$ Lemma 14.1A is proved. \medskip Lemma~14.1B will be proved with the help of the following version Lemma~14.3B of Lemma~14.3A. \medskip\noindent {\bf Lemma 14.3B.} {\it Let us consider a class of functions $\Cal F$ satisfying the conditions of Proposition~13.3 and the random variables $\bar I_{n,k}^V(f,y)$, $f\in\Cal F$, $V\subset\{1,\dots,k\}$, defined in formula (14.3). Let $\Cal B=\Cal B(\xi_1^{(j,1)}, \dots, \xi_n^{(j,1)};\;1\le j\le k)$ denote the $\sigma$-algebra generated by the random variables $\xi_1^{(j,1)},\dots,\xi_n^{(j,1)}$, $1\le j\le k$, i.e. by the random variables with second argument 1 in their upper index taking part in the definition of the random variables $\bar I_{n,k}^V(f,y)$ and $H_{n,k}^V(f)$ introduced in formulas (14.3) and (14.4). \medskip \item{a)} For all $V\in\{1,\dots,k\}$, $V\neq\{1,\dots,k\}$, there exists a number $A_0=A_0(k)>0$ such that the inequality $$ P\(\sup_{f\in\Cal F} E(H^{V}_{n,k}(f)|\Cal B) >2^{-(4k+4)}A^{(2k-1)/k} n^{2k}\sigma^{2k+2}\)< n^{k-1}e^{-A^{1/2k} n\sigma^2/k}. \tag14.24 $$ holds for all $A\ge A_0$. \medskip \item{b)} Given two subsets $V_1,V_1\subset\{1,\dots,k\}$ of the set $\{1,\dots,k\}$ define the integrals of random expressions $$ H_{n,k}^{(V_1,V_2)}(f)=\int |\bar I_{n,k}^{V_1}(f,y) \bar I_{n,k}^{V_2}(f,y)| \rho(\,dy), \quad f\in\Cal F, \tag14.25 $$ with the help of the functions $\bar I_{n,k}^V(f,y)$ defined in (14.3). If at least one of the sets $V_1$ and $V_2$ is not the set $\{1,\dots,k\}$, then there exists some number $A_0=A_0(k)>0$ such that if the integrals $H_{n,k}(f)$, $f\in\Cal F$, determined by this class of functions $\Cal F$ have a good tail behaviour at level $T^{(2k+1)/2k}$ for some $T\ge A_0$, then the inequality $$ P\(\sup_{f\in\Cal F} E(H^{(V_1,V_2)}_{n,k}(f)|\Cal B) >2^{-(2k+2)}A^2n^{2k}\sigma^{2k+2}\)<2n^{k-1}e^{-A^{1/2k}n\sigma^2/k} \tag14.26 $$ holds for all $A\ge T$.} \medskip\noindent {\it Proof of Lemma 14.3B.}\/ Part a) of Lemma 14.3B can be proved in almost the same way as Lemma 14.3A. Hence I only briefly explain the main step of the proof. In the case $V=\emptyset$ $E(H^{V}_{n,k}(f)|\Cal B)=E(H^{V}_{n,k}(f))$, hence it is enough to show that $E(H^{V}_{n,k}(f))\le\frac{n^{k}\sigma^2}{k!}\le \frac{n^{2k}\sigma^{2k+2}}{k!}$ for all $f\in\Cal F$ under the conditions of Proposition~13.3. (Here we exploit in particular that the functions of the class $\Cal F$ are canonical.) The case of a general set $V$, $V\neq\emptyset$ can be reduced to the case $V=\{1,\dots,u\}$ with some $1\le u\frac {A^{(2k-1)/k}n^{k+u}\sigma^{2k+2}}{2^{(4k+4)}}\)\\ &\qquad \le e^{-A^{(2k-1)/2(2u+1)k}(n-u)\sigma^2}. \endalign $$ This inequality can be proved, similarly to relation (14.19) in the proof of Lemma 14.3A with the help of the Corollary of Proposition~13.3. Only here we have to work in the space $(X^u\times \bar Y,\Cal X^u\times\bar{\Cal Y}, \mu^u\times\bar \rho)$ where $\bar Y=X^{k-u}\times Y$, $\bar{\Cal Y}=\Cal X^{k-u}\times\Cal Y$, $\bar\rho=\mu^{k-u}\times\rho$ with the class of function $\Cal F$ so that we identify a function $f(x_1,\dots,x_k,y)\in \Cal F$ with $f(x_1,\dots,x_u,\bar y)=f(x_1,\dots,x_k,y)$ so that $\bar y=(x_{u+1},\dots,x_k,y)$. I omit the details. \medskip\noindent Part b) of Lemma 14.3B will be proved with the help of Part a) and the inequality $$ \sup_{f\in\Cal F} E(H^{(V_1,V_2)}_{n,k}(f)|\Cal B) \le \(\sup_{f\in\Cal F} E(H^{V_1}_{n,k}(f)|\Cal B)\)^{1/2} \(\sup_{f\in\Cal F} E(H^{V_2}_{n,k}(f)|\Cal B)\)^{1/2} $$ which follows from the Schwarz inequality applied for integrals with respect to conditional distributions. Let us assume that $V_1\neq\{1,\dots,k\}$. The last inequality implies that $$ \aligned &P\(\sup_{f\in\Cal F} E(H^{(V_1,V_2)}_{n,k}(f)|\Cal B) >2^{-(2k+2)}A^2n^{2k}\sigma^{2k+2}\)\\ &\qquad \le P\(\sup_{f\in\Cal F} E(H^{V_1}_{n,k}(f)|\Cal B) >2^{-(4k+4)}A^{(2k-1)/k} n^{2k}\sigma^{2k+2}\) \\ &\qquad\qquad+P\(\sup_{f\in\Cal F} E(H^{V_2}_{n,k}(f)|\Cal B) >A^{(2k+1)/k} n^{2k}\sigma^{2k+2}\) \endaligned $$ Hence the estimate (14.24) together with the inequality $$ P\(\sup_{f\in\Cal F} E(H^{V_2}_{n,k}(f)|\Cal B) >A^{(2k+1)/k} n^{2k}\sigma^{2k+2}\)\le n^{k-1} e^{A^{1/2k}n\sigma^2} \tag14.27 $$ imply relation (14.26). Relation 14.27 follows from Part a) of Lemma~14.3B if $V_2\neq\{1,\dots,n\}$ and $A\ge A_0$ with a sufficiently large number~$A_0$ (in this case the level $A^{(2k+1)/k} n^{2k}\sigma^{2k+2}$ can be replaced by the larger number $2^{-(4k+2)}A^{(2k-1)/k} n^{2k}\sigma^{2k+2}$ in the probability of formula (14.27)) and from the conditions of Part~b) of Lemma~14.3B if $V_2=\{1,\dots,k\}$. Indeed, in this case we may apply the estimate (13.5) for this probability, since $A^{(2k+1)/2k}\ge T^{(2k+1)/2k}$, and this implies relation (14.27). \medskip Now we turn to the proof of Lemma~14.1B. \medskip\noindent {\it Proof of Lemma 14.1B.}\/ By Lemma~14.2B it is enough to prove that relation (14.9) holds if the random variables $\bar W(f)$ are replaced in it by the random variables $W(f)$ defined in formula~(14.15). We shall prove this by applying the generalized form of the symmetrization lemma, Lemma~13.1 with the choice of $Z(f)=H_{n,k}^{(\bar V,\bar V)}(f)$, $\bar V=\{1,\dots,k\}$, $\bar Z(f)=W(f)-Z(f)$, $f\in\Cal F$, $\Cal B=\Cal B(\xi_1^{(j,1)},\dots,\xi_n^{(j,1)};\;1\le j\le k)$, and the set $$ B=\bigcap\Sb (V_1,V_2)\: V_j\in \{1,\dots,k\},\;j=1,2\\ V_1\neq\{1,\dots,k\} \text { or } V_2\neq\{1,\dots,k\} \endSb \left\{\oo\: \sup_{f\in\Cal F} E(H^{(V_1,V_2)}_{n,k}(f)|\Cal B)(\oo) \le 2^{-(2k+2)} A^{2} n^{2k}\sigma^{2k+2}\right\}. $$ By Lemma 14.3B the inequality $1-P(B)\le2^{k+1}n^{k-1} e^{A^{1/2k}n\sigma^2/k}$ holds, and to prove Lemma 14.1B with the help of Lemma~13.1 it is enough to show that $$ P\(\left.|\bar Z(f)|>\frac{A^2}2 n^{2k}\sigma^{2(k+1)}\right| \Cal B\)(\oo)\le\frac12 \quad \text{for all }f\in\Cal F\text{ if } \oo\in B. $$ To prove this relation observe that $$ E (|\bar Z(f)| |\Cal B)\le \summ \Sb (V_1,V_2)\: V_j\in \{1,\dots,k\},\;j=1,2\\ V_1\neq\{1,\dots,k\} \text { or } V_2\neq\{1,\dots,k\} \endSb E(H^{(V_1,V_2)}_{n,k}(f)|\Cal B)\le\frac{A^2}4n^{2k}\sigma^{2k+2} \quad \text{if } \oo\in B $$ for all $f\in \Cal F$. Hence the `conditional Markov inequality' implies that $$ P\(\left.|\bar Z(f)|> \frac{A^2}2n^{2k}\sigma^{2k+2}\right|\Cal B\)\le \frac12 \quad\text{if }\oo\in B\quad \text{and } f\in\Cal F. $$ Lemma~14.1B is proved. \beginsection 15. The proof of the main result In this section we prove Proposition 13.2 and thus complete the proof of the main result of this work, of Theorem 8.4 or of its equivalent version Theorem 8.2. Proposition~13.2 will be proved with the help of the symmetrization Lemma~14.1A. In the proof of this symmetrization lemma we have also applied the corollary of Proposition~13.3 (for orders $u2^{-(k+1)}A n^k\sigma^{k+1}\)$ appearing in the estimate (14.2) of Lemma~14.1A. To estimate this probability we introduce (using the notation of Proposition~13.2) the functions $$ S^2_{n,k}(f)(x_l^{(j)},\,1\le l\le n,\,1\le j\le k)=\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k,\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb f^2\(x_{l_1}^{(1)},\dots,x_{l_k}^{(k)}\),\quad f\in\Cal F, \tag15.1 $$ with $x_l^{(j)}\in X$, $1\le l\le n$, $1\le j\le k$. Then we estimate the probability we are interested in with the help of this quantity similarly to the argument applied in the solution of the corresponding problem in the proof of Proposition~6.2. Fix some number $A>T$ and define the set $H\subset X^{kn}$ $$ \aligned H=H(A)&=\biggl\{(x_l^{(j)},\,1\le l\le n,\,1\le j\le k)\:\\ &\qquad \sup_{f\in\Cal F} S^2_{n,k}(f)(x_l^{(j)},\,1\le l\le n,\, 1\le j\le k)>2^kA^{4/3}n^k\sigma^2\biggr\}. \endaligned \tag15.2 $$ We want to show that $$ P(\{\oo\:(\xi_l^{(j)}(\oo),\,1\le j\le n,\,1\le j\le k)\in H\})\le 2^k e^{-A^{2/3k}n\sigma^2} \quad\text{if }A\ge T. \tag15.3 $$ Relation (15.3) will be proved by means of the Hoeff\-ding decomposition (Theorem~9.1) of the $U$-statistics with kernel functions $f^2(x_1,\dots,x_k)$, $f\in\Cal F$, and by the estimation of the sum this decomposition yields. More explicitly, write (applying formula~(9.2) in Theorem 9.1) $$ f^2(x_1,\dots,x_k)=\summ_{V\subset\{1,\dots,k\}} f_V(x_j,j\in V) \tag15.4 $$ with $f_V(x_j,j\in V)=\prodd_{j\notin V}P_j\prodd_{j\in V}Q_j f^2(x_1,\dots,x_k)$, where $P_j$ is the projection defined in formula (9.1) and $Q_j=I-P_j$ is also the same operator as the operator $Q_j$ in formula~(9.2). The functions $f_V$ appearing in formula (15.4) are canonical (with respect to the measure $\mu$), and the identity $S^2_{n,k}(f)(\xi_l^{(j)} \,1\le l\le n,1\le j \le k)=\bar I_{n,k}(f^2)$ holds for all $f\in \Cal F$. By applying the Hoeff\-ding decomposition (15.4) for each term $f^2(\xi_{l_1}^{(1)}\dots,\xi_{l_k}^{(k)})$ in the expression $I_{n,k}(f^2)$ we get that $$ \aligned &P\(\sup_{f\in\Cal F}S^2_{n,k}(f)(\xi_l^{(j)},\,1\le l\le n,\,1\le j\le k) >2^kA^{4/3}n^k\sigma^2\)\\ &\qquad \le\summ_{V\subset\{1,\dots,k\}} P\(\frac{|V|!}{k!} \sup_{f\in\Cal F} n^{k-|V|}|\bar I_{n,|V|}(f_V)|>A^{4/3}n^k\sigma^2\) \endaligned \tag15.5 $$ with the functions $f_V$ in (15.4). We want to give a good estimate for all terms in the sum at the right-hand side in (15.5). For this goal first we show that the classes of functions $\{f_V\: f\in \Cal F\}$ satisfy the conditions of Proposition 13.2 for all $V\subset\{1,\dots,k\}$. The functions $f_V$ are canonical for all $V\subset\{1,\dots,k\}$. It follows from the conditions of Proposition~13.2 that $|f^2(x_1,\dots,x_k)|\le 2^{-2(k+1)}$ and $$ \int f^4(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)\le 2^{-(k+1)}\sigma^2. $$ Hence relations (9.4) and $(9.4')$ of Theorem~9.2 imply that $\left|\supp_{x_j\in X,j\in V}f_V(x_j,j\in V)\right|\le 2^{-(k+2)}\le2^{-(k+1)}$ for all $V\subset\{1,\dots,k\}$ and $\int f^2_V(x_j,j\in V)\prodd_{j\in V}\mu(\,dx_j)\le 2^{-(k+1)} \sigma^2\le\sigma^2$ for all $V\subset\{1,\dots,k\}$. Finally, to check that the class of functions $\Cal F_V=\{f_V\:f\in\Cal F\}$ is $L_2$-dense with exponent $L$ and parameter $D$ observe that for all probability measures $\rho$ on $(X^k,\Cal X^k)$ and pairs of functions $f,g\in \Cal F$ $\int(f^2-g^2)^2\,d\rho\le 2^{-2k}\int(f-g)^2\,d\rho$. This implies that if $\{f_1,\dots,f_m\}$, $m\le D\e^{-L}$, is an $\e$-dense subset of $\Cal F$ in the space $L_2(X^k,\Cal X^k,\rho)$, then the set of functions $\{2^kf_1^2,\dots,2^kf_m^2\}$ is an $\e$-dense subset of the class of functions $\Cal F'=\{2^kf^2\: f\in \Cal F\}$, hence $\Cal F'$ is also an $L_2$-dense class of functions with exponent~$L$ and parameter~$D$. Then by Theorem~9.2 the class of functions $\Cal F_V$ is also $L_2$-dense with exponent $L$ and parameter~$D$ for all sets $V\subset\{1,\dots,k\}$. For $V=\emptyset$, the function $f_V$ is constant, $f_V=\int f^2(x_1,\dots,x_k) \mu(\,dx_1)\dots\mu(\,dx_k)\le\sigma^2$ holds, and $I_{|V|}(f_{|V|})|=f_V\le\sigma^2$. Therefore the term corresponding to $V=\emptyset$ in the sum at the right-hand side of (13.5) equals zero if $A_0\ge1$ in the conditions of Proposition~13.2. I claim that the terms corresponding to sets $V$, $1\le|V|\le k$, in these sums satisfy the inequality $$ \allowdisplaybreaks \align &P\(\sup_{f\in\Cal F}|\bar I_{n,|V|}(f_V)|>A^{4/3}n^{|V|}\sigma^2\)\\ &\qquad \le P\(\sup_{f\in\Cal F} |\bar I_{n,|V|}(f_V)|>A^{4/3}\frac{k!}{|V|!}n^{|V|}\sigma^{|V|+1}\) \le e^{-A^{2/3k}n\sigma^2} \quad\text{if } 1\le|V|\le k. \tag15.6 \endalign $$ The first inequality in (15.6) holds, since $\sigma^{|V|+1}\le\sigma^2$ for $|V|\ge1$, the second inequality follows from the inductive hypothesis if $|V|2^{-(k+2)}A n^{k/2}\sigma^{k+1}\)$ with respect to the random variables $\xi_l^{(j)}$, $1\le l\le n$, $1\le j\le k$ we get with the help of the multivariate version of Hoeff\-ding's inequality (Theorem~12.3) that $$ \align &P\(\left.\left|\bar I_{n,k}^{\e}(f)\right| >2^{-(k+2)}A n^k\sigma^{k+1}\right|\xi_l^{(j)}(\oo)=x_l^{(j)}, 1\le l\le n,1\le j\le k\) \\ &\qquad \le C\exp\left\{-B\(\frac{A^2n^{2k}\sigma^{2(k+1)}}{2^{2k+4} S^2_{n,k}(x_l^{(j)},1\le l\le n,\,1\le j\le k)/k!}\)^{1/k}\right\} \tag15.7 \\ &\qquad \le Ce^{-2^{-3-4/k}BA^{2/3k}k!n\sigma^2} \quad \text{for all }f\in\Cal F\quad \text{if } (x_l^{(j)},\, 1\le l\le n,\,1\le j\le k) \notin H. \endalign $$ Given some points $x_l^{(j)}\in X$, $1\le l\le n$, $1\le j\le k$, define the probability measures $\rho_j=\rho_{j,\,(x_l^{(j)},\, 1\le l\le n)}$, $1\le j\le k$, uniformly distributed on the set $x_l^{(j)}$, $1\le l\le n$, i.e.\ let $\rho_j(x_l^{(j)})=\frac1n$, $1\le l\le n$. Let us also define the product $\rho=\rho_1\times\cdots\times\rho_k$ of these measures. If $f$ is a function on $(X^k,\Cal X^k)$ such that $\int f^2 \,d\rho\le\delta^2$ with some $\delta>0$, then $$ |f(x_{l_j}^{(j)},\,1\le j\le k)|\le\delta n^{k/2}\quad \text{ for all vectors } (l_1,\dots,l_k),\; 1\le l_j\le n,\; 1\le j\le k, $$ and this implies that $P\(\left.\left|\bar I_{n,k}^{\e}(f)\right|>\delta n^{3k/2} \right|\xi_{l}^{(j)}=x_{l}^{(j)},\, 1\le l\le n,\, 1\le j\le k\)=0$ for such a function~$f$. Take the numbers $\bar\delta=An^{-k/2}2^{-(k+2)}\sigma^{k+1}$ and $\delta=2^{-(k+2)}n^{-k-1/2}\!\le\bar\delta$. (The inequality $\delta\le\bar\delta$ holds, since $A\ge A_0\ge1$, and $\sigma\ge n^{-1/2}$.) Choose a $\delta$-dense set $\{f_1,\dots,f_m\}$ in the $L_2(X^k,\Cal X^k,\rho)$ space with $m\le D\delta^{-L}\le 2^{(k+2)L} n^{\beta+(k+1)L/2}$ elements. Then the above estimates, the $\delta$-dense property of the set of functions $\{f_1,\dots,f_m\}$ in $L_2(X^k,\Cal X^k,\rho)$ and formula (15.7) imply that $$ \align &P\(\sup_{f\in\Cal F}\left.\left|\bar I_{n,k}^{\e}(f)\right| >2^{-(k+1)}A n^k\sigma^{k+1}\right|\xi_l^{(j)}(\oo)=x_l^{(j)}, 1\le l\le n,1\le j\le k\) \\ &\quad \le \sum_{s=1}^m P\(\left.\left|\bar I_{n,k}^{\e}(f_s)\right| >2^{-(k+2)}A n^k\sigma^{k+1}\right|\xi_l^{(j)}(\oo)=x_l^{(j)}, 1\le l\le n,1\le j\le k\) \tag15.8 \\ &\qquad \le C 2^{(k+2)L}n^{\beta+(k+1)L/2} e^{-2^{-3-4/k}BA^{2/3k}nk!\sigma^2} \quad \text{if }\{x_l^{(j)},\, 1\le l\le n,\,1\le j\le k\}\notin H. \endalign $$ Relations (15.3) and (15.8) imply that $$ \aligned &P\(\sup_{f\in\Cal F}\left|\bar I_{n,k}^{\e}(f)\right| >2^{-(k+1)}A n^k\sigma^{k+1}\) \\ &\qquad \le C2^{(k+2)L}n^{\beta+(k+1)L/2} e^{-2^{-3-4/k}BA^{2/3k}n\sigma^2}+ 2^k e^{-A^{2/3k}k!n\sigma^2} \quad\text{if }A\ge T. \endaligned \tag15.9 $$ Proposition 13.2 follows from the estimates (14.2) and (15.9) if the constant $A_0$ together with the constant $K$ in the condition $n\sigma^2\ge K(L+\beta) \log n$ are chosen sufficiently large. In this case these estimates yield an upper bound less than $e^{-A^{1/2k}n\sigma^2}$ for the probability at the left-hand side of (13.3). Now we turn to the proof of Proposition~13.3. \medskip\noindent {\script B.) The proof of Proposition 13.3.} \medskip\noindent Because of formula (14.12) in the corollary of Lemma~14.1B to prove Proposition 13.3 i.e. inequality (13.5) it is enough to choose the parameter $A_0$ in Proposition~13.3 for which $A>T\ge A_0$ sufficiently large and to show that $$ \aligned &P\(\sup_{f\in\Cal F} \left |H_{n,k}(f|G,V_1,V_2)\right| >\frac{A^2}{2^{4k+1}k!} n^{2k}\sigma^{2(k+1)}\) \le 2^{k+1} e^{-A^{1/2k}n\sigma^2}\\ &\qquad\text{ for all } G\in \Cal G\quad \text{and } \;V_1,V_2\in\{1,\dots,k\} \quad\text{if } A\ge A_0 \endaligned \tag15.10 $$ with the random variables $H_{n,k}(f|G,V_1,V_2)$ defined in formula (14.10). Let us first prove formula (15.10) in the case when $|e(G)|=k$, i.e.\ when all vertices of the diagram $G$ are end-points of some edge, and the expression $H_{n,k}(f|G,V_1,V_2)$ contains no `symmetryzing term' $\e_j$. In this case we apply a special argument to prove relation~(15.10). If $G$ is such a diagram for which $|e(G)|=k$, then we can show by means of the Schwarz inequality that $$ \aligned |H_{n,k}(f|G,V_1,V_2)|&\le\frac1{k!} \(\sum\Sb l_1,\dots,l_k,\, 1\le l_j\le n,\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \int f^2(\xi_{l_1}^{(1),\delta_1)},\dots,\xi_{l_k}^{(k,\delta_k)},y) \rho(\,dy)\)^{1/2}\\ &\qquad \frac1{k!}\(\sum\Sb l_1,\dots,l_k, 1\le l_j\le n,\\ l_j\neq l_{j'} \text{ if }j\neq j'\endSb\int f^2(\xi_{l_1}^{(1,\bar\delta_1)}, \dots,\xi_{l_k}^{(k,\bar\delta_k)},y) \rho(\,dy)\)^{1/2}, \endaligned \tag15.11 $$ where $\delta_j=1$ if $j\in V_1$, $\delta_j=-1$ if $j\notin V_1$, and $\bar\delta_j=1$ if $j\in V_2$, $\bar\delta_j=-1$ if $j\notin V_2$. Indeed, in this case the sum of integrals in (14.10) can be rewritten in a natural way as the integral of the product of two functions on the product space $(I_n^k\times Y,\Cal I_n^k\times\Cal Y,\lambda_n^k \times\rho)$, where $I_n=\{1,\dots,n\}$, $\Cal I_n$ is the $\sigma$-algebra of all subsets of $I_n$, and $\lambda_n$ is the counting measure on $\Cal I_n$. Then the Schwarz inequality for this product yields formula (15.11). (Observe that the coordinates $l_1,\dots,l_k$ determine the coordinates $l'_1,\dots,l'_k$ in the summation (14.10) if $|e(G)|=k$.) By formula (15.11) $$ \align &\left\{\oo\:\sup_{f\in\Cal F} \left |H_{n,k}(f|G,V_1,V_2)(\oo)\right| >\frac{A^2}{2^{4k+1}k!} n^{2k}\sigma^{2(k+1)}\right\} \\ &\;\; \subset \left\{\oo\:\sup_{f\in\Cal F} \! \sum\Sb l_1,\dots,l_k, 1\le l_j\le n,\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \! \int f^2(\xi_{l_1}^{(1,\delta_1)}(\oo),\dots,\xi_{l_k}^{(k,\delta_k)} (\oo),y) \rho(\,dy)>\frac {A^2n^{2k}\sigma^{2(k+1)}k!} {2^{4k+1}} \right\}\\ &\;\; \cup \left\{\oo\:\sup_{f\in\Cal F} \! \sum\Sb l_1,\dots,l_k, 1\le l_j\le n,\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \! \int f^2(\xi_{l_1}^{(1,\bar\delta_1)}(\oo),\dots,\xi_{l_k}^{(k,\bar\delta_k)} (\oo),y) \rho(\,dy)>\frac{A^2n^{2k}\sigma^{2(k+1)}k!}{2^{4k+1}}\right\}, \endalign $$ and $$ \aligned &P\(\sup_{f\in\Cal F} \left |H_{n,k}(f|G,V_1,V_2)\right| >\frac{A^2}{2^{4k+1}k!} n^{2k}\sigma^{2(k+1)}\) \\ &\qquad \le 2P\(\sup_{f\in\Cal F}\frac1{k!} \sum\Sb l_1,\dots,l_k, 1\le l_j\le n,\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb h_f(\xi_{l_1}^{(1,1)},\dots,\xi_{l_k}^{(k,1)}) >\frac{A^2n^{2k}\sigma^{2(k+1)}}{2^{4k+1}}\) \endaligned \tag15.12 $$ with the functions $h_f(x_1,\dots,x_k)=\int f^2(x_1,\dots,x_k,y)\rho(\,dy)$, $f\in\Cal F$. (In this upper bound we could get rid of the terms $\delta_j$ and $\bar\delta_j$, i.e. on the dependence of the expression $H_{n,k}(f|G,V_1,V_2)$ on the sets $V_1$ and $V_2$, since the probability of the events in the previous formula do not depend on these terms.) I claim that $$ P\(\supp_{f\in\Cal F} |\bar I_{n,k}(h_f)|\ge An^k \sigma^2\)\le 2^k e^{-A^{1/2k}n\sigma^2} \quad \text{for }A\ge A_0 \tag15.13 $$ if the constants $A_0$ and $K$ are chosen sufficiently large in Proposition~13.3. Relation (15.13) together with the relation $A\frac{n^{2k}\sigma^{2(k+1)}}{2^{4k+1}}\ge n^k\sigma^2$ imply that the probability at the right-hand side of (15.12) can be bounded by $2^{k+1}e^{-A^{1/2k}n\sigma^2}$, and the estimate~(15.10) holds in the case $|e(G)|=k$. Relation (15.13) is similar to relation (15.5), and the proof of the latter formula helps to carry out the proof in the present case. Indeed, it follows from the conditions of Proposition~13.3 that $0\le\int h_f(x_1,\dots,x_k) \mu(\,dx_1)\dots\mu(\,dx_k)\le\sigma^2$, and it is not difficult to check that$\sup|h_f(x_1,\dots,x_k)|\le 2^{-2(k+1)}$, and the class of functions $\Cal H=\{2^kh_f,\; f\in\Cal F\}$ is an $L_2$-dense class with exponent $L$ and parameter $D$. This means that by applying the Hoeff\-ding decomposition of the functions $h_f$, $f\in \Cal F$, similarly to formula (15.4) we get such sets of functions $(h_f)_V$, $f\in\Cal F$, for all $V\subset \{1,\dots,k\}$ which satisfy the conditions of Proposition~13.2. Hence a natural adaptation of the estimate given for the expression at the right-hand side of (15.5) yields the proof of formula (15.13). We only have to replace $S_{n,k}(f)$ by $I_{n,k}(h_f)$, $I_{n,|V|}(f_V)$ by $I_{n,|V|}((h_f)_V)$ and the levels $2^kA^{4/3}n^k\sigma^2$ and $A^{4/3}n^k\sigma^2$ by $An^k\sigma^2$ and $2^{-k}An^k\sigma^2$. Let us observe that each term of the upper bound we get in such a way can be directly bounded, since our inductive hypothesis the result of Proposition~13.2 holds also for $k$. In the case $e(G)A^{8/3}n^{2k}\sigma^4\)\le 2^{k+1}e^{-A^{2/3k}n\sigma^2} \quad\text{if }A\ge A_0\text{ and } e(G)A^{8/3}n^{2k}\sigma^4) \le 2P\(\supp_{f\in\Cal F} \bar I_{n,k}(h_f)>A^{4/3}n^k\sigma^2\) $$ with $h_f(x_1,\dots,x_k)=\int f^2(x_1,\dots,x_k,y)\rho(\,dy)$. (Here we exploited that in the last formula $S^2(\Cal F|G,V_1,V_2)$ is bounded by the product of two random variables whose distributions do not depend on the sets $V_1$ and $V_2$.) Thus to prove inequality (15.15) it is enough to show that $$ 2P\(\supp_{f\in\Cal F} \bar I_{n,k}(h_f)>A^{4/3}n^k\sigma^2\)\le 2^{k+1}e^{-A^{2/3k}n\sigma^2} \quad \text{if } A\ge A_0. \tag15.18 $$ Actually formula (15.18) follows from the already proven formula (15.13), only the parameter $A$ has to be replaced by $A^{4/3}$ in it. With the help of relation (15.15) the proof of Proposition~13.3 can be completed similarly to that of Proposition~13.2. It follows from the generalized version of Hoeff\-ding's inequality Theorem~12.3 and the definition of the random variable $H_{n,k}(f|G,V_1,V_2)$ given in the form (15.14) that $$ \aligned &P\(\left.|H_{n,k}(f|G,V_1,V_2)| >\frac{A^2}{2^{4k+2}k!} n^{2k}\sigma^{2(k+1)} \right| \xi^{j,\pm1}_{l},\,1\le l\le n,\,1\le j\le k\)(\oo)\\ &\qquad \le Ce^{-B2^{-(4+2/k)} A^{2/3k}n\sigma^2} \quad \text{if}\quad S^2(\Cal F|G,V_1,V_2)(\oo)\le A^{8/3}n^{2k}\sigma^4 \quad\text{ for all } f\in\Cal F, \\ &\qquad\text{ and } G\in \Cal G\text{ such that } |e(G)|\frac{A^2}{2^{4k+1}k!} n^{2k}\sigma^{2(k+1)} \right| \xi^{j,\pm1}_{l},\,1\le l\le n,\,1\le j\le k\)(\oo)\\ &\qquad \le Cn^{(3k+1)L/2+\beta} e^{-BA^{2/3k}n\sigma^2/2^{(4+2/k)}(k!)^{1/k}} \quad \text{if } S^2(\Cal F|G,V_1,V_2))(\oo)\le A^{8/3}n^{2k}\sigma^4 \\ &\qquad \qquad\text{ for all } G\in \Cal G\text{ such that }|e(G)|0$. (This bound does not depend on the `randomizing terms' $\e_l(\oo)$ in the definition of the random variable $H_{n,k}(\cdot|G,V_1,V_2)$.) In this case $\int(f-g)^2\,d\alpha_j\le2\delta^2$, and $$ \align \int&|f(x_{l_1}^{(1,\delta_1)},\dots,x_{l_k}^{(k,\delta_k)},y)- g(x_{l_1}^{(1,\delta_1)},\dots,x_{l_k}^{(k,\delta_k)},y)|^2 \rho(\,dy) \le2\delta^2n^k, \\ \int& |f(x_{l_1}^{(1,\delta_1)},\dots,x_{l_k}^{(k,\delta_k)},y)- g(x_{l_1}^{(1,\delta_1)},\dots,x_{l_k}^{(k,\delta_k)},y)| \rho(\,dy) \le\sqrt2\delta n^{k/2} \endalign $$ for all $1\le l\le k$, and $1\le l_j\le n$, and the same result holds if all $\delta_j$ is replaced by $\bar\delta_j$, $1\le j\le k$. Since $|f|\le1$, $|g|\le1$ if $f,g\in\Cal F$, the condition $\int(f-g)^2\,d\alpha\le \delta^2$ implies that $$ \align \int &|f(\xi_{l_1}^{(1,\delta_1)}(\oo),\dots, \xi_{l_k}^{(k,\delta_k)}(\oo),y) f(\xi_{l'_1}^{(1,\bar\delta_1)}(\oo),\dots, \xi_{l'_k}^{(k,\bar\delta_k)}(\oo),y) \rho(\,dy)\\ &\qquad -g(\xi_{l_1}^{(1,\delta_1)}(\oo),\dots, \xi_{l_k}^{(k,\delta_k)}(\oo),y) g(\xi_{l'_1}^{(1,\bar\delta_1)}(\oo),\dots, \xi_{l'_k}^{(k,\bar\delta_k)}(\oo),y) \rho(\,dy)| \le2\sqrt2\delta n^{k/2} \endalign $$ for all vectors $(l_1,\dots,l_k,l'_1,\dots,l'_k)$ which appear as an index in the summation in (14.10). Hence $$ |H_{n,k}(f|G,V_1,V_2)(\oo)-H_{n,k}(g|G,V_1,V_2)(\oo)| \le2\sqrt2\delta n^{5k/2} $$ if $f,g\in \Cal F$, $\int(f-g)^2\,d\alpha<\delta^2$ and the originally fixed $\oo\in\Omega$ is considered. (The measure $\alpha$ is defined by means of this $\oo$.) Put $\bar\delta=\frac{A^2 n^{-k/2}\sigma^{2(k+1)}}{2^{(4k+7/2)} k!}$, and $\delta=n^{-(3k+1)/2}\le\bar\delta$ ( the inequality $\delta\le\bar\delta$ holds, since $\sigma\ge \frac1{\sqrt n}$ and we may assume that $A\ge A_0$ is sufficiently large), choose a $\delta$-dense subset $\{f_1,\dots,f_m\}\subset\Cal F$ in the $L_2(X^k\times Y,\Cal X^k\times\Cal Y,\alpha)$ space with $m\le D\delta^{-L}\le n^{(3k+1)L/2+\beta}$ elements. Relation (15.19) for these functions together with the above estimates yield formula (15.20). It follows from relations (15.15) and (15.20) that $$ \align &P\(\sup_{f\in\Cal F}|H_{n,k}(f|G,V_1,V_2)| >\frac{A^2}{2^{4k+1}k!} n^{2k}\sigma^{2(k+1)}\)\le 2^{k+1}e^{-A^{2/3k}n\sigma^2}\\ &\qquad + Cn^{(3k+1)L/2+\beta} e^{-B2^{-(4+2/k)}A^{2/3k}n\sigma^2} \quad\text{if }A\ge A_0 \endalign $$ for all $V_1,V_2\subset\{1,\dots,k\}$ also in the case $|e(G)|\le k-1$. This inequality implies that relation (15.10) holds also in this case if the constants $A_0$ and $K$ are chosen sufficiently large in Proposition~13.3. Proposition~13.3 is proved. \beginsection 16. The improvement of some results in Section 8 In this section I present an improvement of Theorems~$8.3$ and~8.5. I shall explain the picture behind these results and the some ideas of the proofs. But the detailed proofs which are based on some results called the diagram formulas for the products of multiple Wiener--It\^o integrals and degenerate $U$-statistics are omitted. These diagram formulas present an identity which enables us to express the product of Wiener--It\^o integrals or degenerate $U$-statistics as a sum of such objects. I omitted the proofs because they heavily depend on the diagram formula, a technique not discussed in this work. The interested reader can find the detailed proofs in my papers~[21] and~[22]. The main result discussed in this section is the following \medskip\noindent {\bf Theorem 16.1.} {\it Let $\xi_1,\dots,\xi_n$ be a sequence of iid. random variables on a space $(X,\Cal X)$ with some distribution~$\mu$. Let us consider a function $f(x_1,\dots,x_k)$ canonical with respect to the measure~$\mu$ on the space $(X^k,\Cal X^k)$ which satisfies conditions (8.1) and (8.2) with some $0<\sigma^2\le1$ together with the degenerate $U$-statistic $I_{n,k}(f)$ with this kernel function. There exist some constants $A=A(k)>0$ and $B=B(k)>0$ depending only on the order $k$ of the $U$-statistic $I_{n,k}(f)$ such that $$ P(k!n^{-k/2}|I_{n,k}(f)|>u)\le A\exp\left\{-\frac{u^{2/k}}{2\sigma^{2/k} \(1+B\(un^{-k/2}\sigma^{-(k+1)}\)^{1/k}\)}\right\} \tag16.1 $$ for all $0\le u\le n^{k/2}\sigma^{k+1}$.} \medskip\noindent Theorem 16.1 states in particular that if $00$, then $P(k!n^{-k/2}|I_{n,k}(f)|>u)\le A\exp\left\{-\frac{1-C\e^{1/k}}2 \(\frac u\sigma\)^{2/k}\right\}$ with some universal constants $A>0$ and $C>0$ depending only on the order~$k$ of the $U$-statistic $I_{n,k}(f)$. This result is very similar to Theorem~8.3. Both theorems yield an estimate on the probability $P(k!n^{-k/2}|I_{n,k}(f)|>u)$ for $0\le u\le n^{k/2}\sigma^{k+1}$, but in the present result we also get a good estimate on the constant $\alpha$ in formula (8.9) for $0\le u\le \e n^{k/2}\sigma^{k+1}$. At first sight this additional result does not seem an essential improvement, but actually it expresses an important property of the estimate (16.1). To understand this it is worth while to compare Theorem~16.1 with Bernstein's inequality formulated in Theorem~3.1. Theorem~3.1 implies the estimate $$ P(n^{-1/2}|I_{n,1}(f)|>u)\le2e^{-Cu^2/\sigma^2} \quad\text{if } 0\le u\le n\sigma^2 \tag16.2 $$ for the degenerate $U$-statistic $I_{n,1}(f)$ of order 1 with a kernel function $f$, (i.e. for a sum of iid. random variables $Ef(\xi_1)=0$) if the relations $\sup |f(x)|\le 1$ and $Ef(\xi_j)=0$ and $Ef^2(\xi_j)\le\sigma^2$ hold. Beside this, relation~(16.2) also holds with a constant of the form $C=\frac{1-O(\e)}2$ if $0\le u\le\e n\sigma^2$. On the other hand, Example~3.2 shows an example (formulated with a different normalization) with a function $f$ and a sequence of iid.\ random variables $\xi_1,\xi_2,\dots$ satisfying the above conditions such that $$ P(n^{-1/2}I_{n,1}(f)>u)\ge A\exp\left\{-B\(\frac u\sigma\)^2\cdot\frac{\sqrt n\sigma^2}u \log\frac u{\sqrt n\sigma^2}\right\} $$ if $u\gg n\sigma^2$. This means that in the special case $k=1$ the probability $P(n^{-1/2}|I_{n,1}(f)|>u)$ has a Gaussian type estimate for $0\le u\le\const n\sigma^2$, and such an estimate does not hold for $u\gg n\sigma^2$. Beside this, in the smaller interval $0\le u\le\e n\sigma^2$ we can say more. In this case the relation (16.2) holds with such a constant $C$ which almost agrees with the number $\frac12$, i.e. the upper bound we get for $k=1$ almost agrees with the quantity suggested by a formal application of the central limit theorem. I want to explain that Theorem~16.1 states a similar result for degenerate $U$-statistics of any order~$k\ge1$. To understand this let us first recall that a sequence of normalized degenerate $U$-statics $ n^{-k/2}I_{n,k}(f)$, $n=1,2,\dots$, defined with the help of a sequence of iid.\ random variables $\xi_1,\xi_2,\dots$ taking values on some measurable space $(X,\Cal X)$ with distribution $\mu$ and a function $f(x_1,\dots,x_k)$ of $k$ variables canonical with respect to $\mu$ and such that $$ \sigma^2=\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)<\infty $$ has a limit distribution as $n\to\infty$. Moreover, this limit can be expressed explicitly as the distribution of the Wiener--It\^o integral $$ Z_{\mu,k}(f)=\frac1{k!}\int f(x_1,\dots,x_k)\mu_W(\,dx_1)\dots \mu_W(dx_k), \tag16.3 $$ where $\mu_W$ is the white noise with counting measure $\mu$, i.e.\ $\mu_W(A)$, $A\in \Cal X$, is a Gaussian field indexed by the measurable subsets of the space $X$ such that $E\mu_W(A)=0$ and $E\mu_W(A)\mu_W(B)=\mu(A\cap B)$ for all $A,B\in\Cal X$. (The definition of Wiener--It\^o integrals can be found e.g. in~[17].) Hence it is natural to expect that in the estimates about the distribution of degenerate $U$-statistics the distributions of Wiener--It\^o integrals play a role similar to the Gaussian distributions in the case $k=1$. Therefore we are interested in good estimates on the distribution of Wiener--It\^o integrals. The next result supplies such an estimate. As Theorem~16.1 was an improvement of Theorem~8.3, the next result is an improvement of the first estimate in~Theorem~8.5 presented in formula~(8.11). \medskip\noindent {\bf Theorem 16.2.} {\it Let us consider a $\sigma$-finite measure $\mu$ on a measurable space together with a white noise $\mu_W$ with counting measure $\mu$. Let us have a real-valued function $f(x_1,\dots,x_k)$ on the space $(X^k,\Cal X^k)$ which satisfies relation (8.2). Take the random integral $Z_{\mu,k}(f)$ introduced in formula (16.3). This random integral satisfies the inequality $$ P(k!|Z_{\mu,k}(f)|>u)\le C \exp\left\{-\frac12\(\frac u\sigma\)^{2/k}\right\}\quad \text{for all } u>0 \tag16.4 $$ with an appropriate constant $C=C(k)>0$ depending only on the multiplicity $k$ of the integral.} \medskip In Theorem 16.2 we gave only an upper bound for the distribution of Wiener--It\^o integrals. The following example shows that there are cases when this estimate is essentially sharp. \medskip\noindent {\bf Example 16.3.} {\it Let us have a $\sigma$-finite measure $\mu$ on some measure space $(X,\Cal X)$ together with a white noise $\mu_W$ on $(X,\Cal X)$ with counting measure~$\mu$. Let $f_0(x)$ be a real valued function on $(X,\Cal X)$ such that $\int f_0(x)^2\mu(\,dx)=1$, and take the function $f(x_1,\dots,x_k)=\sigma f_0(x_1)\cdots f_0(x_k)$ with some number $\sigma>0$ and the Wiener--It\^o integral $Z_{\mu,k}(f)$ introduced in formula (16.3). Then the relation $\int f(x_1,\dots,x_k)^2\,\mu(\,dx_1)\dots\,\mu(\,dx_k)=\sigma^2$ holds, and the random integral $Z_{\mu,k}(f)$ satisfies the inequality $$ P(k!|Z_{\mu,k}(f)|>u)\ge \frac{\bar C}{\(\frac u\sigma\)^{1/k}+1} \exp\left\{-\frac12\(\frac u\sigma\)^{2/k}\right\}\quad \text{for all } u>0 \tag16.5 $$ with some constant $\bar C>0$.} \medskip\noindent {\it Proof of the statement of Example 16.3.}\/ We may restrict our attention to the case $k\ge2$. It\^o's formula (see e.g.~[17]) states that the random variable $k!\bar Z_{\mu,k}(f)$ can be expressed as $k!Z_{\mu,k}(f)=\sigma H_k\(\int f_0(x)\mu_W(\,dx)\)=\sigma H_k(\eta)$, where $H_k(x)$ is the $k$-th Hermite polynomial with leading coefficient~1, and $\eta=\int f_0(x)\mu_W(\,dx)$ is a standard normal random variable. Hence we get by exploiting that the coefficient of $x^{k-1}$ in the polynomial $H_k(x)$ is zero that $P(k!|Z_{\mu,k}(f)|>u)=P(|H_k(\eta)| \ge\frac u\sigma)\ge P\(|\eta^k|-D|\eta^{k-2}|>\frac u\sigma\)$ with a sufficiently large constant $D>0$ if $\frac u\sigma>1$. There exist such positive constants $A$ and $B$ that $P\(|\eta^k|-D|\eta^{k-2}|>\frac u\sigma\) \ge P\(|\eta^k|>\frac u\sigma+A\(\frac u\sigma\)^{(k-2)/k}\)$ if $\frac u\sigma>B$. Hence $$ P(k!|Z_{\mu,k}(f)|>u)\ge P\(|\eta|>\(\frac u\sigma\)^{1/k}\(1+A\(\frac u\sigma\)^{-2/k}\)\) \ge \frac{\bar C \exp\left\{-\frac12\(\frac u\sigma\)^{2/k}\right\}} {\(\frac u\sigma\)^{1/k}+1} $$ with an appropriate $\bar C>0$ if $\frac u\sigma>B$. Since $P(k!|Z_{\mu,k}(f)|>0)>0$, the above inequality also holds for $0\le \frac u\sigma\le B$ if the constant $\bar C>0$ is chosen sufficiently small. This means that relation (16.5) holds. Let us remark that if $f(x_1,\dots,x_k)=\sigma f_0(x_1)\dots f_0(x_k)$ is a function on the space $(X^k,\Cal X^k)$ such that $\int f_0(x)\mu(\,dx)=0$, $\int f_0^2(x)\mu(\,dx)=1$, $\sup |f_0(x)|\le 1$, $0<\sigma\le1$, and we have a sequence of iid. random variables, $\xi_1,\xi_2,\dots$ with distribution $\mu$, then the $U$-statistics $I_{n,k}(f)$, $n=1,2,\dots$, are degenerate, and they satisfy the conditions of Theorem~16.1. Beside this, they converge in distribution to the Wiener--It\^o integral $Z_{\mu,k}(f)$ as $n\to\infty$ which satisfies the conditions of example (16.3). Hence the $U$-statistics $I_{n,k}(f)$ satisfy relation (16.1), and also the inequality $$ \lim_{n\to\infty} P(k!n^{-k/2}|I_{n,k}(f)|>u)\ge \frac{\bar C\exp\left\{-\frac12\(\frac u\sigma\)^{2/k}\right\}} {\(\frac u\sigma\)^{1/k}+1} $$ holds with an appropriate $\bar C>0$ if $\frac u\sigma>B$. This means that for not too large values of $u$, more explicitly if $u\le\e n^{k/2}\sigma^{k+1}$ with a small number $\e>0$, the estimate given in Theorem~16.1 is essentially sharp. Let me also remark that Example~8.6 shows such a degenerate $U$-statistic of order $k=2$ for which an estimate similar to that of Theorem~8.3 cannot hold for $n\gg n^{k/2}\sigma^{k+1}$. We have presented such an example only for $k=2$, but similar examples can be given for all~$k\ge1$. This means that Theorem~16.1 shows a similar picture about the distribution of degenerate $U$-statistics of order~$k$ for all $k\ge1$ as Bernstein's inequality shows in the case $k=1$. We have a good estimate on the distribution $P(n^{-k/2}I_{n,k}(f)>u)$ of a degenerate $U$-statistic with a kernel function $f$ satisfying relations (8.1) and (8.2) in the domain $0\le u\le n^{k/2}\sigma^{k+1}$. Such an estimate is already proved in Theorem~8.3, but Theorem~16.1 says more in an interval of the form $0\le u\le \e n^{k/2}\sigma^{k+1}$ with a small $\e>0$. The limit theorems about degenerate $U$-statistics give an upper bound for the coefficient $\alpha$ in the exponent of formula (8.9) in Theorem~8.3, and Theorem~16.1 states that the estimate (8.9) holds with an almost as good coefficient $\alpha$ in the interval $0\le u\le\e n^{k/2}\sigma^{k+1}$ as this upper bound suggests. The proof of the above results are based, similarly to the proof of Theorems~8.3 and~8.5, on some good estimates on high moments of degenerate $U$-statistics $I_{n,k}(f)$ and of Wiener--It\^o integrals $Z_{n,k}(f)$. The result of Theorem~16.2 follows from the following \medskip\noindent {\bf Proposition 16.4.} {\it Let the conditions of Theorem~16.2 be satisfied for a multiple Wiener--It\^o integral $Z_{\mu,k}(f)$ of order~$k$. Then, with the notations of Theorem~16.2, the inequality $$ E\(k!|Z_{\mu,k}(f)|\)^{2M}\le 1\cdot3\cdot5\cdots (2kM-1)\sigma^{2M}\quad\text {for all }M=1,2,\dots \tag16.6 $$ holds.}\medskip By the Stirling formula Proposition~16.4 implies that $$ E(k!|Z_{\mu,k}(f)|)^{2M}\le \frac{(2kM)!}{2^{kM}(kM)!}\sigma^{2M}\le A\(\frac2e\)^{kM}(kM)^{kM}\sigma^{2M} \tag16.7 $$ for all numbers $A>\sqrt2$ if $M\ge M_0=M_0(A)$. The following Proposition~16.5 states a similar, but weaker inequality for the moments of normalized degenerate $U$-statistics. \medskip\noindent {\bf Proposition 16.5.} {\it Let us consider a degenerate $U$-statistic $I_{n,k}(f)$ of order $k$ with sample size $n$ and with a kernel function $f$ satisfying relations (8.1) and (8.2) with some $0<\sigma^2\le1$. Fix a positive number $\eta>0$. There exists some universal constants $A=A(k)>\sqrt2$, $C=C(k)>0$ and $M_0=M_0(k)\ge1$ depending only on the order of the $U$-statistic $I_{n,k}(f)$ such that $$ \aligned E\(n^{-k/2}k!I_{n,k}(f)\)^{2M}&\le A\(1+C\sqrt\eta\)^{2kM} \(\frac2e\)^{kM}\(kM\)^{kM}\sigma^{2M} \\ &\qquad \text{for all integers } M \text{ such that } kM_0\le kM\le \eta n\sigma^2. \endaligned \tag16.8 $$ The constant $C=C(k)$ in formula (2.3) can be chosen e.g. as $C=2\sqrt2$ which does not depend on the order $k$ of the $U$-statistic $I_{n,k}(f)$.}\medskip Let us remark that formula (16.6) can be reformulated as $E(k!|Z_{\mu,k}(f)|)^{2M}\le E(\sigma\eta^k)^{2M}$, where $\eta$ is a standard normal random variable. Theorem~16.2 states that the tail distribution of $k!|Z_{\mu,k}(f)|$ satisfies an estimate similar to that of $\sigma|\eta|^k$. This follows simply from Proposition~16.4 and the Markov inequality $P(k!|Z_{\mu,k}(f)|>u)\le \frac{E(k!|Z_{\mu,k}(f)|)^{2M}}{u^{2M}}$ with an appropriate choice of the parameter~$M$. Proposition 16.5 states that in the case $M_0\le M\le\e n\sigma^2$ the inequality $$ E\(n^{-k/2}k!I_{n,k}(f)\)^{2M}\le E((1+\beta(\e))\sigma\eta^k)^{2M} $$ holds with a standard normal random variable $\eta$ and a function $\beta(\e)$, $0\le\e\le1$, such that $\beta(\e)\to0$ if $\e\to0$, and $\beta(\e)\le C$ with some universal constant $C=C(k)$ if $0\le\e\le1$. This means that certain high but not too high moments of $n^{-k/2}k!I_{n,k}(f)$ behave similarly to the moments of $k!Z_{\mu,k}(f)$. As a consequence, we can prove a similar, but slightly weaker estimate for the distribution of $n^{-k/2}k!I_{n,k}(f)$ as for the distribution of $k!Z_{\mu,k}(f)$. Actually this is done in the proof of Theorem~16.1. Estimate (16.8) is very similar to the bound (10.1) formulated in Proposition~(10.1). The main difference is that here we get the estimate $$ E\(n^{-k/2}k!I_{n,k}(f)\)^{2M}\le C^M (kM)^{kM}\sigma^{2M} \tag16.9 $$ with a good constant $C$, at least if $M\le\e n\sigma^2$ with a small number $\e>0$. The method of proof of Theorem~8.3 presented in this paper cannot yield such a good estimate. The main problem with this method is that it applies a symmetrization argument (this is done in the proof of the Marcinkiewicz--Zygmund inequality), in which we bound the moments of the random variable we are investigating by the moments of a random variable with constant times larger variance. Such a step in the proof does not allow to get the estimate~(16.9) with a good constant~$C>0$. On the other hand, the estimation of the moments of a degenerate $U$-statistics by means of the diagram formula yields a better estimate of the moments. The idea behind this approach is that in calculating the even moments $E\(I_{n,k}(f)\)^{2M}$ of a degenerate $U$-statistics by means of the diagram formula we have to work with some terms which also appear in the calculation of the moments $E(Z_{\mu,k}(f))^{2M}$ of the Wiener--It\^o integral~$Z_{\mu,k}(f)$, but we also have to handle some additional terms. It must be checked that the contribution of these additional terms is not too large. This is the case if $M\le n\sigma^2$ with $\sigma^2=\int f^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)$, and an even better estimate can be given about the contribution of these terms if $M\ge\e n\sigma^2$ with a small~$\e>0$. Let me finally remark that the above method can also give an improvement of the multivariate version of the Hoeffding inequality (Theorem 12.3). The proof of the following inequality can be found in~[22]. \medskip\noindent {\bf Theorem 16.6. The multivariate version of Hoeffding's inequality.} {\it Let $\e_1$,\dots, $\e_n$ be independent random variables, $P(\e_j=1)=P(\e_j=-1)=\frac12$, $1\le j\le n$. Fix a positive integer~$k$, and define the random variable $$ Z=\sum\Sb (j_1,\dots, j_k)\: 1\le j_l\le n \text{ for all } 1\le l\le k\\ j_l\neq j_{l'} \text{ if }l\neq l' \endSb a(j_1,\dots, j_k) \e_{j_1}\cdots \e_{j_k} \tag16.10 $$ with the help of some real numbers $a(j_1,\dots,j_k)$ which are given for all sets of indices such that $1\le j_l\le n$, $1\le l\le k$, and $j_l\neq j_{l'}$ if $l\neq l'$. Put $$ S^2=\sum\Sb (j_1,\dots, j_k)\: 1\le j_l\le n \text{ for all } 1\le l\le k\\ j_l\neq j_{l'} \text{ if }l\neq l' \endSb a^2(j_1,\dots, j_k) \tag16.11 $$ Then $$ P(k!|Z|>u)\le C \exp\left\{-\frac12\(\frac uS\)^{2/k}\right\} \quad\text{for all }u\ge 0 \tag16.12 $$ with some constant $C>0$ depending only on the parameter $k$.} \medskip\noindent We may assume that the coefficients $a(j_1,\dots,j_k)$ in formulas (16.10) and (16.11) are symmetric functions of their arguments, i.e.\ $a(j_1,\dots,j_k)=a(j_{\pi(1)},\dots,j_{\pi(k)})$ for all permutations $\pi\in\Pi_k$ of the set $\{1,\dots,k\}$. If these coefficients $a(j_1,\dots,j_k)$ do not have not this symmetry property, then we can replace them with their symmetrizations $a_{\text{Sym}}(j_1,\dots,j_k)=\frac1{k!}\summ_{\pi\in \Pi_k} a(j_{\pi(1)},\dots,j_{\pi(k)}$. In such a way we do not modify the value of the random variable~$Z$, and decrease the value of the number $S^2$. With such a choice of the coefficients we have $EZ=0$ and $\Var Z=k!S^2$. The main advantage of this result with respect to Theorem~12.3 is that formula (16.12) holds with the right constant in the exponent at the right-hand side. The proof is based on good moment estimates of the random variable $Z$ defined in (16.10). I formulate this result which may be interesting in itself. \medskip\noindent {\bf Theorem 16.7} {\it The random variable $Z$ defined in formula (16.10) satisfies the inequality $$ EZ^{2M}\le 1\cdot3\cdot5\cdots(2kM-1)S^{2M}\quad\text{for all } M=1,2,\dots \tag16.13 $$ with the constant $S$ defined in formula (16.11).} \medskip It is worth while to compare formula (16.13) with the estimate that Borell's inequality yields for this problem. By applying Borell's inequality with the choice $q=2$ and $p=2M$ we get that $EZ^{2M}\le (2M-1)^{kM}E(Z^2)^M=(2M-1)^{kM}\(k!S^2\)M$. Since $(2M-1)^{2M}= (2M)^{kM}\(1-\frac1{2M}\)^{kM}\sim e^{-k/2} (2M)^{kM}$ for large values~$M$, hence Borell's inequality yields the inequality $EZ^{2M}\le\const(2M)^{kM}S^{2M}\cdot (k!)^M$ for large exponents~$M$. On the other hand, Theorem~16.7 together with the Stirling formula yield the estimate $EZ^{2M}\le\const (2M)^{kM}S^{2M}\cdot\(\frac ke\)^{kM}$. It can be seen that $k!>\(\frac ke\)^k$ for all $k\ge1$. This means that Theorem~16.7 yields an improvement of the Borell's inequality in the special case discussed above. This estimate is only a special case of Borell's inequality, but this is its most important special case. \beginsection 17. An overview of the results in this work I discuss briefly the problems investigated in this work and recall some basic results related to them. I also give a list of works where they can be found. Beside this, I discuss some background problems and results which may explain the motivation for the study presented here. I met the main problem considered in this work when tried to adapt the method of proof of the central limit theorem for maximum-likelihood estimates to some more difficult questions about so-called non-parametric maximum likelihood estimate problems. The Kaplan--Meyer estimate for the empirical distribution function with the help of censored data investigated in the second section is such a problem. It is not a maximum-likelihood estimate in the classical sense, but it can be considered as a non-parametric maximum likelihood estimate. Indeed, since in the estimation of a distribution function with the help of censored data the class of possible candidates for being the distribution function we are looking for is too large, there is no dominating measure with respect to which all of them have a density function. As a consequence, the classical principle of the maximum-likelihood estimate cannot be applied in this case. A natural way to overcome this difficulty is to choose a smaller class of distribution functions, to compare the probability of the appearance of the sample we observe with respect to all distribution functions of this class and to choose that distribution function as our estimate for which this probability takes its maximum. The Kaplan--Meyer estimate can be found on the basis of this principle in the following way: Let us estimate the distribution function $F(x)$ of the censored data simultaneously with the distribution function $G(x)$ of the censoring data. (We have a sample of size $n$ and know which sample elements are censored and which are censoring data.) Let us consider the class of such pairs of estimates $(F_n(x),G_n(x))$ of the pair $(F(x),G(x))$ for which the distribution function $F_n(x)$ is concentrated in the censored sample points and the distribution function $G_n(x)$ is concentrated in the censoring sample points; more precisely, let us also assume that if the largest sample point is a censored point, then the distribution function $G_n(x)$ of the censoring data takes still another value which is larger than any sample point, and if it is a censoring point then the distribution function $F_n(x)$ of the censored data takes still another value larger than any sample point. (This modification at the end of the definition is needed, since if the largest sample points is from the class of censored data, then the distribution $G(x)$ of the censoring data in this point must be strictly less than~1, and if it is from the class of censoring data, then the value of the distribution function $F(x)$ of the censored data must be strictly less than~1 in this point.) Let us take this class of pairs of distribution functions $(F_n(x),G_n(x))$, and let us choose that pair of distribution functions of this class as the (non-parametric maximum likelihood) estimate with respect to which our observation has the greatest probability. The above extremal problem for the pairs of distribution functions $(F_n(x),G_n(x))$ can be solved explicitly, and it yields the estimate of $F_n(x)$ written down in formula~(2.3). (The function $G_n(x)$ satisfies a similar relation, only the random variables $X_j$ and $Y_j$ and the events $\delta_j=1$ and $\delta_j=0$ have to be replaced in it.) Then, as I have indicated, a natural analog of the linearization procedure in the maximum likelihood estimate also works in this case, and there is only one really hard part of the proof. We need a good estimate on the distribution of the integral of a function of two variables with respect to the product of a normalized empirical measure with itself. Moreover, we also need a good estimate on the distribution of the supremum of a class of integrals, when the elements of an appropriate class of functions are integrated with respect to the above product measure. The main subject of this work is to solve the above problems in a more general setting, when not only two-fold, but also $k$-fold integrals are considered with arbitrary number~$k\ge1$. The proof of this work for the limit behaviour of the Kaplan--Meyer estimate applied the explicit form of this estimate. It would be interesting to find such a modification of this proof which exploits that the Kaplan--Meyer estimate is the solution of an appropriate extremal problem. We may expect that such a proof can be generalized to a general result about the limit behaviour for a wide class of non-parametric maximum likelihood estimates. Such a consideration is behind the remark of Richard Gill I quoted at the end of Section~2. I hope that such a program can be realized, but at the present time I cannot do this. A detailed proof together with a sharp estimate on the speed of convergence for the limit behaviour of the Kaplan--Meyer estimate based on the ideas presented in Section~2 is given in paper [24]. Paper [25] explains more about its background, and it also discusses the solution of some other non-parametric maximum likelihood problems. The results about multiple integrals with respect to a normalized empirical distribution function needed in these works were proved in~[17]. The results of~[18] are completely satisfactory for the study in~[24], but they also have some drawbacks. They do not show that if the random integrals we are considering have small variances, then they satisfy better estimates. Beside this, if we consider the supremum of random integrals of an appropriate class of functions, then these results can be applied only in very special cases. Moreover, the method of proof of~[18] did not allow a real generalization of its results, hence I had to find a different approach when tried to generalize them. I do not know of other works where the distribution of multiple random integrals with respect to a normalized empirical distribution is studied. On the other hand, there are some works where the distribution of (degenerate) $U$-statistics is investigated. The most important results obtained in this field are contained in the book of de la Pe\~na and Gin\'e {\it Decoupling, From Dependence to Independence}\/~[6]. The problems about the behaviour of degenerate $U$-statistics and multiple integrals with respect to a normalized empirical distribution function are closely related, but the explanation of their relation is far from trivial. I return to this question later. Even the study of the one-dimensional version of the problems studied here, i.e. the description of the behaviour of one-fold integrals or classes of one-fold integrals contains several hard problems which have to be investigated closely to have a good understanding of the subject. In the one-dimensional case it is fairly simple to prove that the problems about the behaviour of one-fold integrals with respect to a normalized empirical measure and about the behaviour of normalized sums of independent random variables are equivalent. I start this work with the description of the case of (classes of) one-fold integrals or of sums of independent random variables. This question has a fairly big literature. I would mention first of all the books {\it A course on empirical processes}\/~[9], {\it Real Analysis and Probability}\/~[10] and {\it Uniform Central Limit Theorems}\/~[11] written by R.~M.~Dudley. These books contain a much more detailed description of the empirical processes than the present work together with a lot of interesting results. The first problem studied here deals with the tail behaviour of sums of independent and bounded random variables with expectation zero. This question is considered in Section~3 where the proof of two already classical results, that of Bernstein's and Bennett's inequalities is explained. (These results are proved e.g.~[4]). We are also interested in the question when these results give an estimate suggested by the central limit theorem. Bernstein's inequality provides such an estimate if the variance of the sum is not too small. (The results in Section~3 tell explicitly when this variance should be considered too small.) If the variance of the sum is too small, then Bennett's inequality provides a slight improvement of Bernstein's inequality. On the other hand, Example~3.2 shows that in the unpleasant case when this variance is too small Bennett's inequality is essentially sharp. I inserted this example to the text, because it may help to understand better the content of Bernstein's and Bennett's inequality. I have not found similar examples in the literature. The estimate on the distribution of a sum of independent random variables if this sum has a small variance is weak because of the following reason. In this case the probability that the sum will be larger than a given value may be much larger than the (rather small) value suggested by the central limit theorem because of the appearance of some irregularities with relatively large probability. The hardest problems we have to cope with in the solution of the problems of this work are closely related to the weak estimates for sums of independent random variables if the variance of the sums are small and to the weak estimates in some similar problems. The weakness of these estimates imply that in the study of the problems we are interested in the method of proof for their Gaussian counterpart cannot be adapted completely, some new ideas are needed. We have overcome this difficulty by applying a symmetrization argument. The last result of Section~3, Hoeff\-ding's inequality presented in Theorem~3.4 is an important ingredient of this symmetrization argument. It is also a classical result whose proof can be found for instance in~[15]. In Section~4 I formulated the one-variate version of our main result about the supremum of the integrals of a class $\Cal F$ of functions with respect to a normalized empirical measure together with an equivalent statement about the distribution of the supremum of a class of random sums $\summ_{j=1}^nf(\xi_j)$ defined with the help of a sequence of i.i.d. random variables $\xi_1,\dots,\xi_n$ and a class of functions $f\in\Cal F$ satisfying some appropriate conditions. These results are given in Theorems~4.1 and~$4.1'$. Also a Gaussian version of them is presented in Theorem~4.2 about the distribution of the supremum of a Gaussian field with some appropriate properties. In the above mentioned results we have imposed the condition that the class of functions~$\Cal F$ or what is equivalent the set of random variables whose supremum we estimate is countable. In the proofs this condition is really exploited. On the other hand, in some important applications we also need results about the supremum of a possibly non-countable set of random variables. Hence I introduced the notion of countably approximable classes of random variables and proved that in the results of this work the condition about countability can be replaced by the weaker condition that the class of random variables whose supremum is taken is countably approximable. R.~M.~Dudley worked out a different method to handle the supremum of possibly non-countably many random variables, and generally his method is applied in the literature. The relation between these two methods deserves some discussion. Let us first recall that if a class of random variables $S_t$, $t\in T$, indexed by some index set $T$ is given, then a set $A$ can be measurable with respect to the $\sigma$-algebra generated by the random variables $S_t$, $t\in T$, only if there exists a countable subset $T'=T'(A)\subset T$ such that the set $A$ is also measurable with respect to the smaller $\sigma$-algebra generated by the random variable $S_t$, $t\in T'$. Beside this, if the finite dimensional distributions of the random variables $S_t$, $t\in T$, are given, then by the results of classical measure theory also the probability of the events measurable with respect to the $\sigma$-algebra generated by these random variables $S_t$, $t\in T$, is determined. But there are rather few other events whose probabilities are determined by the finite dimensional distributions of the random variables~$S_t$, $t\in T$. On the other hand, if $T$ is a non-countable set, then the events $\left\{\supp_{t\in T}S_t>u\right\}$ are not measurable with respect to the above $\sigma$-algebra, hence generally we cannot speak of their probabilities. To overcome this difficulty Dudley worked out a theory which enabled him to work also with outer measures. His theory is based on some rather deep results of the analysis. It can be found for instance in his book~[11]. I restricted my attention to the case when after the completion of the probability measure $P$ we can also speak of the real (and not only outer) probabilities $P\(\supp_{t\in T}S_t>u\)$. I tried to find appropriate conditions under which these probabilities really exist. More explicitly, we are interested in the case when for all $u>0$ there exists some set $A=A_u$ measurable with respect to the $\sigma$-algebra generated by the random variables $S_t$, $t\in T$, such that the symmetric difference of the sets $A_u$ and $\left\{\supp_{t\in T}S_t>u\right\}$ is contained in a set measurable with respect to the $\sigma$-algebra generated by the random variables $S_t$, $t\in T$, and has probability zero. In such a case we can define also the probability $P\(\supp_{t\in T}S_t>u\)$ as $P(A_u)$. This approach led me to the definition of countable approximable classes of random variables. Its validity enables us to speak about the probability of the event that the supremum of the random variables we are interested in is larger than some fixed value. I also proved a simple but useful result in Lemma~4.3, which provides a condition for the validity of this property. The problem we met here is not an abstract, technical difficulty. Indeed, the distribution of such a supremum can become different if we modify each random variable at a set of probability zero, although the joint distribution of the random variables we consider remains the same after such an operation. Hence, if we are interested in the supremum of a non-countable set of random variables with described joint distribution we have to describe more explicitly which version of this set of random variables we consider. It is natural to look for such an appropriate version of the random field $S_t$, $t\in T$, whose `trajectories' $S_t(\oo)$, $t\in T$, have nice properties for all elementary events $\oo\in\Omega$. Lemma~4.3 can be interpreted as a result in this spirit. The condition given for the countable approximability of a class of random variables at the end of this lemma can be considered as a smoothness condition about the `trajectories' of the random field we consider. This approach shows some analogy to some important problems in the theory of stochastic process when a regular version of a stochastic process is considered and the smoothness properties of its trajectories are investigated. In our problems the version of the set of random variables $S_t$, $t\in T$, we shall work with appears in a simple and natural way. In these problems we have finitely many random variables $\xi_1,\dots,\xi_n$ at the start, and all random variables $S_t(\oo)$, $t\in T$, we are considering can be defined individually for each $\oo$ as a functional of these random variables $\xi_1(\oo),\dots,\xi_n(\oo)$. We take the version of the random field $S_t(\oo)$, $t\in T$, we get in such a way and want to show that it is countably approximable. In Section~4 we have proved this property in an important model, probably in the most important model in possible applications we are interested in. In more complicated situations when our random variables are defined not as a functional of finitely many sample points, for instance in the case when we define our set of random variables by means of integrals with respect to a Gaussian field it is harder to find the right regular version of our sets of random variables. In this case the integrals we consider are defined only with probability~1, and we have to make some extra work to find their right version. At any rate, in the problems we are interested in our approach is satisfactory for our purposes, and it is simpler than that of Dudley; we do not have to follow his rather difficult technique. On the other hand, I must admit that I do not know the precise relation between the approach of this work and that of Dudley. In Section~4 the notion of $L_p$-dense classes, $1\le p<\infty$, is also introduced. The notion of $L_2$-dense classes plays an important role in the formulation Theorems~4.1 and~$4.1'$. The notion of $L_2$-dense classes can be considered as a version of the $\e$-entropy discussed at many places in the literature. On the other hand, there seems to be no unique definition of $\e$-entropy in the literature. I introduced the term of $L_2$-dense classes, because this seems to be the appropriate notion in the study of this work. To apply the results related to $L_2$-dense classes we also need some knowledge about how to check it in concrete models. For this goal I discussed here Vapnik--\v{C}ervonenkis classes, a popular and important notion of modern probability theory. Several books and papers, (see e.g. the books~[11], [28],~[30] and the references in them) deal with this subject. An important result in this field is Sauer's lemma, (Lemma~5.2) which together with some other results, like Lemma~5.3 imply that the classes of sets or functions are in many several interesting models Vapnik--\v{C}ervonenkis classes. I put these results to the Appendix, partly because they can be found in the literature, partly because in our investigation Vapnik--\v{C}ervonenkis classes play a different and less important role than at other places. In our discussion Vapnik--\v{C}ervonenkis classes are applied to show that certain classes of functions are $L_2$-dense. A result of Dudley formulated in Lemma~5.2 implies that a Vapnik--\v{C}ervonenkis class of functions with absolute value bounded by a fixed constant is an $L_1$, hence also an $L_2$-dense class of functions. The proof of this important result which seems to be less known even among experts of this subject than it should be is contained in the main text. Dudley's original result was formulated in the special case when the functions we consider are indicator functions of some sets, but its proof contained all important ideas needed in the proof of Lemma~5.2. Theorem 4.2, which is the Gaussian counterpart of Theorems~4.1 and~$4.1'$ is proved in Section~6 by means of a natural and important technique, called the chaining argument. We apply an inductive procedure, during which an appropriate sequence of finite subsets of our set of random variables is defined, and try to give a good estimate on the supremum of these subsets of our random variables. The subsets we consider are denser and denser subsets of the original set of random variables, and if they are constructed in a clever way, then we get the result we want to prove by means of a limiting procedure. In such a way we get a relatively simple proof of Theorem 4.2, but this method is not strong enough to supply a complete proof of Theorem~4.1. The cause of the weakness of the method in this case is that we cannot give a good estimate on the probability that a sum of independent random variables is greater than a prescribed value if these random variables have too small variances. The chaining argument supplies a result much weaker than that what we want to prove under the conditions of Theorem~4.1. Lemma~6.1 contains the result the chaining argument yields under the conditions of Theorem~4.1. In Section~6 still another result, Lemma~6.2 is formulated, and it is also shown that Lemmas~6.1 and~6.2 together imply Theorem~4.1. The proof is not difficult, despite of some non-attractive details. We have to check that the parameters in Lemmas~6.1 and~6.2 can be fitted to each other. Lemma~6.2 is proved in Section~7. It is based on a symmetrization argument. This proof applies the ideas of a paper of Kenneth Alexander~[1], and although its presentation is essentially different of Alexander's approach, it can be considered as a version of his proof. A similar problem should also be mentioned at this place. M.~Talagrand wrote a series of papers about concentration inequalities, and this research was also continued by some other authors. I would mention the works of M.~Ledoux~[16] and P.~Massart~[26]. Concentration inequalities give a bound about the difference of the supremum of a set of appropriately defined random variables from its expected value; they express how strongly this supremum is concentrate around its expected value. Such results are closely related to Theorem~4.1, and the discussion of their relation deserves some attention. A typical concentration inequality is the following result of Talagrand~[29]. \medskip\noindent {\bf Theorem 17.1. (Theorem of Talagrand.)} {\it Consider $n$ independent and identically distributed random variables $\xi_1,\dots,\xi_n$ with values in some measurable space $(X,\Cal X)$. Let $\Cal F$ be some countable family of real-valued measurable functions of $(X,\Cal X)$ such that $\|f\|_\infty\le b<\infty$ for every $f\in\Cal F$. Let $Z=\supp_{f\in\Cal F}\summ_{i=1}^n f(\xi_i)$ and $v=E(\supp_{f\in\Cal F}\summ_{i=1}^n f^2(\xi_i))$. Then for every positive number~$x$, $$ P(Z\ge EZ+x)\le K\exp\left\{-\frac 1{K'}\frac xb\log\(1+\frac{xb}v\)\right\} $$ and $$ P(Z\ge EZ+x)\le K\exp\left\{-\frac{x^2}{2(c_1v+c_2bx)}\right\}, $$ where $K$, $K'$, $c_1$ and $c_2$ are universal positive constants. Moreover, the same inequalities hold when replacing $Z$ by $-Z$.} \medskip Theorem~17.1 yields, similarly to Theorem~4.1, an estimate about the distribution of the supremum for a class of sums of independent random variables. It can be considered as a generalization of Bernstein's and Bennett's inequalities when the distribution of the supremum of partial sums is estimated. A remarkable feature of this result is that it assumes no condition about the structure of the class of functions $\Cal F$ (like the condition of $L_2$-dense property of the class $\Cal F$ imposed in Theorem~4.1.) On the other hand, the estimates in Theorem~17.1 contain the quantity $EZ=E\(\supp_{f\in\Cal F}\summ_{i=1}^n f(\xi_i)\)$. Such an expectation of some supremum appears in all concentration inequalities. As a consequence, they are useful only if we can bound the expected value of an appropriate supremum. This is a hard question in the general case, and this is the reason why I preferred a direct proof of Theorem~4.1 without the application of concentration inequalities. Let me remark that the condition $u\ge\const\sigma\log^{1/2}\frac2\sigma$ with some appropriate constant which cannot be dropped from Theorem~4.1 is related to the fact that the expected value of the supremum of the normalized random sums considered in Theorem~4.1 has such a magnitude. The main results of this work are presented in Section~8. Theorem~8.3 which contains an estimate about the distribution of a degenerate $U$-statistic was first proved in a paper of Gin\'e and Arcones in~[2], its equivalent version about the multiple integrals with respect to a normalized empirical measure formulated in Theorem~8.1 in my paper~[19]. The equivalence of these two results is not self-evident. Later I proved an improved version of Theorem~8.3 in paper~[21]. This result is formulated in Theorem~16.1, and it is also compared with Theorem~8.3. It is also explained that Theorem~16.1 could be considered the multivariate version of Bernstein's inequality with more right than Theorem~8.3. Here I omitted its proof which applies a technique (diagram formulas for the calculation of products of multiple random integrals or degenerate $U$-statistics) not discussed in this work. Here Theorem~8.3 was proved by means of a symmetrization argument. The explanation of such a proof was simpler in the present work, because it applies such methods which were worked out in the investigation of other problems. On the other hand, some arguments can be posed against such a proof. The application of symmetrization arguments in the proof of Theorem~8.3 also has some drawbacks. In certain problems, like the problem of Theorem~8.3, this method cannot supply a really sharp result. Some mathematicians working in this field seem not to be aware of this fact. It may be interesting to mention that the problem of Theorem~8.3 has a natural generalization worth of a closer study. We can consider such generalized $U$-statistics in which the underlying random variables $\xi_1,\dots,\xi_n$ are independent, but they need not be identically distributed, and the $U$-statistic also may have a more general form. Namely, we can take a class of kernel functions $\bold f=\{f_{l_1,\dots,l_k}(x_1,\dots,x_k)\}$ on the space $(X^k,\Cal X^k)$ with such an indexation that $1\le l_j\le n$, $1\le j\le k$, and $l_j\neq l_{j'}$ if $j\neq j'$, and define with the help of these independent random variables and class of kernel functions the generalized $U$-statistic $$ I_{n,k}(\bold f)=\sum\Sb 1\le l_j\le n,\; 1\le j\le k\\ l_j\neq l_{j'} \text{ if }j\neq j'\endSb f_{l_1,\dots,l_k}(\xi_{l_1},\dots,\xi_{l_k}). \tag17.1 $$ One can also naturally define generalized degenerate $U$-statistics. We call a generalized $U$-statistic degenerate if for all sets of indices $(l_1,\dots,l_k)$ in the sum (17.1) and for all $1\le j\le k$ $$ E(f_{l_1,\dots,l_k}(\xi_{l_1},\dots,\xi_{l_k})|\xi_{l_s},\; s\in \{1\dots, k\}\setminus\{j\})\equiv0. $$ Generalized degenerate $U$-statistics can be considered as the natural multivariate generalizations of sums of independent random variables, just as degenerate $U$-statistics are the natural multivariate generalizations of sums of iid.\ random variables. One would also try to generalize Theorem~8.3 to an estimation about the distribution of generalized degenerate $U$-statistics. One may hope that the method of proof of Theorem~8.3 can also be applied for the study of generalized degenerate $U$-statistics, just as the distribution of sums independent random variables can be investigated similarly to the sums of iid. random variables. Probably, the methods worked out for the study of the problems related to Theorem~8.3 are helpful, but in the study of generalized degenerate $U$-statistics first some special questions have to be clarified. We have to find the right form of the estimation about the distribution of a generalized degenerate $U$-statistic. In particular, it must be clarified which are the natural quantities by which we should express this estimate. It is natural to expect that generalized degenerate $U$-statistics $I_{n,k}(\bold f)$ of order~$k$ (without normalization) satisfy the inequality $$ P(|I_{n,k}(\bold f)|>u)0$ and $C=C(k)>0$ in a relatively large interval for the parameter~$u$, where $V^2_n$ denotes the variance of $I_{n,k}(\bold f)$. An essential problem is to find a relatively good constant $C$ and to determine the interval $0u\)$ can be bounded for a degenerate $U$-statistic $I_{n,2}(f)$ of order~2 by the estimate given in Theorem~8.3 only if $u\le\const n\sigma^3$, i.e. this condition of Theorem~8.3 (in the case $k=2$) cannot be dropped. Similar examples could be constructed for all $k\ge1$. The paper of Arcones and Gin\'e~[2] contains another example explained by Talagrand to the authors which also has a similar consequence. On the other hand, this example does not exclude the possibility to prove such a multi-dimensional version of Hoeffding's inequality Theorem~3.3 which provides a slight improvement of Theorems~8.1 and~8.3 similarly to the improvement of Bernstein's inequality provided by Hoeffding's inequality. Moreover, we can also expect such a strengthened form of Theorems~8.2 and~8.4 (or of Theorem~4.2 in the one-dimensional case) which takes into account the above improvements if the supremum of a nice class of random integrals or degenerated $U$-statistics is considered. There is a hope that some refinement of the methods of the present work would supply such results. However, here we did not study this problem. Theorems~9.2 and~9.3 deal with the properties of degenerate $U$-statistics. This subject deserves special attention. Degenerate $U$-statistics can be considered as the multivariate version of sums of independent and identically distributed random variables with expectation zero. Similarly, if $f$ is a canonical function with respect to a measure $\mu$ and put independent $\mu$-distributed random variables into its arguments, then the random variables we get in such a way can be considered as the multivariate version of random variables with expectation zero. The background of several proofs about the behaviour of $U$-statistics can be better understood with the help of the above remark. I tried to explain for instance that the proof about the Hoeff\-ding decomposition of $U$-statistics (Theorem~9.1) is actually a natural adaptation of the decomposition of a random variable to the sum of a random variable with expectation zero plus the expected value of the random variable. Hoeff\-ding's decomposition is a fairly well-known result which can be found for instance in the Appendix of~[12]. Theorem~9.1 slightly differs from the formulation of Hoeff\-ding's decomposition one usually meets in the literature. It can be exploited that a $U$-statistic does not change if we replace its kernel function by its symmetrized version. Beside this, the value of the $U$-statistics $I_{n,|V|}(f_V)$ do not change if we replace the kernel function $f_V(x_{j_1},\dots,x_{j_{|V|}})$, $V=\{j_1,\dots,j_{|V|}\}$, by $f_V(x_1,\dots,x_{|V|})$ in the Hoeffding decomposition (9.3) of the $U$-statistic $I_{n,k}(f)$, and $f_V(x_1,\dots,x_{|V|})$ is also a canonical function. The above observations enable us to unify the contribution of all terms $I_{n,|V|}(f_V)$ with $|V|=l$ for some $0\le l\le k$ into one non-degenerate $U$-statistics of order $l$. Generally, the formula obtained in such a way is called the Hoeff\-ding decomposition in the literature. Nevertheless, we have applied Theorem~9.1 in this work, because this form of the Hoeffding's decomposition was more convenient for us. In our investigations it is important to know that if a function satisfies a good $L_2$-norm or $L_\infty$-norm estimate, then the elements of its Hoeff\-ding decomposition also have this property, and if a class of function is $L_2$-dense, then the same relation holds for the classes of functions in the Hoeff\-ding decomposition of the functions in this class. This is the content of Propositions~9.2 and~9.3. The estimates on the $L_2$-norm given in formulas~(9.7) and~(9.8) are actually reformulations of some well-known facts about the properties of conditional expectations. Theorem~9.4 enables us to reduce the estimates about multiple random integrals with respect to normalized empirical measures to estimates about degenerate $U$-sta\-tis\-tics. Such random integrals are actually sums of $U$-statistics, and we can apply for each of these $U$-statistics the Hoeff\-ding decomposition. Beside this, as we consider multiple integrals with respect to a {\it normalized}\/ empirical measure we can expect that a lot of cancellations appear during the calculation by which we express our random integral in the form of linear combination of degenerate $U$-statistics. We get such a representation which enables us to reduce the estimates we want to prove about multiple random integrals to analogous estimates about degenerate $U$-statistics. This is the main content of Theorem~9.4 which can be considered as an analog of the Hoeff\-ding decomposition for multiple stochastic integrals with respect to normalized empirical measures. This representation of a multiple stochastic integral as a linear combination of degenerate $U$-statistics of different order also contains degenerate $U$-statistics of low order. But as a consequence of the cancellation effects these $U$-statistics are multiplied with small coefficients. The proof of Theorem~9.4 is based on a good ``book-keeping'' of the different contributions to the integral $J_{n,k}(f)$. An essential, although less spectacular step of this ``book-keeping'' procedure is to express the terms we are working with by means of the (signed) measures $\mu$ and $\mu^{(l)}-\mu$, i.e. the measures $\mu^{(l)}$ have to be replaced by their normalizations $\mu^{(l)}-\mu$. The calculations needed in the proof are quite natural, but unfortunately they contain some unpleasant and complicated technical details. Theorem~9.4 also has the consequence that the second moment of the multiple random integral of a function with respect to a normalized empirical measure can be bounded by constant times the $L_2$-norm of the kernel function we integrate. The representation of the stochastic integrals given in Theorem~9.4 may also contain a non-zero constant term. This has the unexpected consequence that the expected value of a multiple random integral with respect to a normalized empirical measure can be non-zero. Our random integrals may show such an unusual behaviour because the numbers of sample points falling to disjoint sets are not independent random variables. But the dependence between such random variables is very weak, and the expected value of the random integrals we consider is sufficiently small. From the pair of Theorems~8.1 and~8.3 I have proved only Theorem 8.3, since its proof is simpler, and by the results of Section~9 Theorem~8.1 follows from it. The proof of Theorem~8.3 is different from its original proof published in paper~[2]. First a good estimate is presented about the moments of the degenerate $U$-statistics in Proposition 10.1. Theorem~8.3 can be deduced from this estimate. Actually the proof is different, first a version Theorem~$8.3'$ of Theorem~8.3 is proved, where an analogous estimate is proved for degenerate decoupled $U$-statistic. The adjective `decoupled' refers to the fact that we put independent copies of a sequence of iid.\ random variables in different coordinates of the kernel function of the $U$-statistic. The study of decoupled $U$-statistics is a popular subject of some authors. In particular, the main subject of the book~[6] is a comparison of the properties of $U$-statistics and decoupled $U$-statistics. The study of decoupled $U$-statistics is simpler than that of usual $U$-statistics, because the arguments applied in the study of usual $U$-statistics can be applied for them, and beside this they also satisfy a multivariate version of the Marcinkiewicz--Zygmund inequality. On the other hand, the Marcinkiewicz--Zygmund inequality does not hold for usual $U$-statistics, at least the proofs I know of do not work for them. We can prove with the help of the multivariate version of the Marcinkiewicz--Zygmund and Borell's inequality an estimate about the moments of degenerate $U$-statistics formulated in Proposition~$10.1'$. Proposition~$8.3'$ can be deduced from Proposition~$10.1'$, and by a result of de la Pe\~na and Montgomery--Smith formulated in Theorem~10.4 Theorem~$8.3'$ implies Theorem~8.3. The results applied in the proof of Theorem~8.3 are proved in Section~11. Let me also remark that Proposition~10.1 is not proved in this text, since we chose such an approach where we do not need it. On the other hand, it follows from the results of this work and some other standard results about $U$-statistics not discussed in the present work. I have mentioned the possibility of another proof of Theorem~8.3 on the basis of the methods of the theory of Wiener--It\^o integrals to this problem. In~[19] I gave a proof of Theorem~8.1 by means of the so-called diagram method. Let me also remark that the method of paper~[21] which yields an improvement of Theorem~8.3 presented in Theorem~16.1 is actually a refinement of the method in~[19]. Both in paper~[19] and in the present work the main step of the proof consists of finding a good estimate on the moments of the random variables we are investigating. It is enough to estimate the moments of the type $M=2^m$, where $m$ is a positive integer. For $m=1$ such an estimate is known, and we can get an estimate for $m>1$ by means of a recursive procedure. A similar approach is applied in~[19] and in the present work. The main difference between them is in the form of the recursive inequality between the moments of the random variables we work with and the way we prove them. I found the result about the multivariate version of the Marcinkiewicz--Zygmund equation in the book~[6], but the proof of the result given here is different. Only the proof about the upper estimate of the $p$-th moment of decoupled $U$-statistics is written down. There is also an estimate in the opposite direction, but such a result would be interesting for us only for the sake of some orientation. Theorem~10.4 was proved by de la Pe\~na and Montgomery--Smith in their paper~[7]. I formulated their result for separable Banach space valued random variables, just as they did it. Such a general formulation of the results is very popular in the literature, but here the discussion of Banach space valued random variables had a different cause. I also wanted to prove formula $(10.8')$, a result which is actually not contained in paper~[7]. (Book~[6] contains this result, but the proof is left to the reader.) The simplest way to get this statement was to prove the original result in Banach spaces, and to apply it in appropriate $L_\infty$ spaces. Paper~[7] also contains some kind of converse result of~Theorem~10.4, but as we do not need it I omitted its discussion. This work contains the proof of de la Pe\~na and Montgomery--Smith for Theorem~10.4, but I have explained it in my own style. In particular, I worked out some details where the author gave only a very short explanation. This proof is given in the Appendix. The proof of Borell's inequality is closely related to that of Nelson's inequality. Edward Nelson published the inequality named after him in his paper~[27]. He also showed that the general inequality presented in Appendix~C can be reduced to the inequality given in formula~(C1) or in Proposition~C2 of this work. This reduction follows actually from our Theorem~11.2. However, this observation did not help him to find a proof, and finally he gave a proof without its application. Borell's inequality can also be reduced to a one-dimensional statement formulated in Theorem~11.3. This seems to be a simple inequality, but its proof is surprisingly hard. Actually in this paper it is enough to prove this inequality in the special case $q=2$ and $p=2k$, $k=1,2,\dots$. Actually, as I mentioned in Theorem~16.6, Borell's inequality can be proved in this special case with better constants. (See paper~[22].) In the proof of Theorem~11.3 I have followed the paper of Leonhard Gross {\it Logarithmic Sobolev inequalities}\/~[13]. Gross has worked out a general theory and he could prove both Nelson's and Borell's inequality (more precisely an estimate which simply implies this result) with its help. Gross' method and results are interesting, because they are very useful in several parts of the mathematics. (See e.g [16] or~[14].) Let me also remark that similar results and ideas also appeared in an earlier work of A.~Bonami~[5]. Gross introduced a so-called logarithmic Sobolev inequality related to Markov processes and showed that it implies another inequality, which is in the case of a Wiener process Nelson's inequality, while we can define such a simple Markov process for which the logarithmic Sobolev inequality corresponding to it yields the proof of Theorem~11.3. This Markov process is explicitly described in Section~11, and the logarithmic Sobolev inequality corresponding to it is also formulated and proved there. Actually Gross showed that each logarithmic Sobolev inequality is equivalent to the inequality he proved as its consequence. On the other hand, the proof of the logarithmic Sobolev inequalities is less difficult than a direct proof of the inequalities he has obtained as their consequence. The name `logarithmic Sobolev inequality' has the following explanation. Generally one calls `Sobolev inequality' such inequalities where for some pairs of numbers $1\le qu\)\le AP\(\|\bar I_{n,k}(f(\ell))\|>\gamma u\) \tag10.8b $$ with some constants $A=A(k)$ and $\gamma=\gamma(k)$ depending only on the order of these $U$-statistics. To prove relation (10.8b) first we verify the following statement. Let us take two independent copies $\xi_{1}^{(1)},\dots,\xi_n^{(1)}$ and $\xi_{1}^{(2)},\dots,\xi_n^{(2)}$ of our original sequence of random variables $\xi_1,\dots,\xi_n$ and introduce for all sets $V\subset \{1,\dots,k\}$ the function $\alpha_V(j)$, $1\le j\le k$, defined as $\alpha_V(j)=1$ if $j\in V$ and $\alpha_V(j)=2$ if $j\notin V$. Let us define with the help of these quantities the decoupled generalized $U$-statistics $$ I_{n,k,V}(f(\ell))=\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \!\!\!\! f_{l_1,\dots,l_k} \(\xi_{l_1}^{(\alpha_V(1))},\dots,\xi_{l_k}^{(\alpha_V(k))}\) \quad \text{for all }V\subset \{1,\dots,k\}. \tag B1 $$ The following inequality will be proved: There are some constants $C_k>0$ and $D_k>0$ depending only on the order $k$ of the generalized $U$-statistic $I_{n,k}(f(\ell))$ such that for all numbers $u>0$ $$ P\(\|I_{n,k}(f(\ell))\|>u\)\le \sum_{V\subset\{1,\dots,k\},\,1\le|V|\le k-1} C_kP\(D_k\| I_{n,k,V}(f(\ell))\|>u\). \tag B2 $$ Here $|V|$ denotes the cardinality of the set $V$, and the condition $1\le |V|\le k-1$ in the summation of formula (B2) means that we omit the sets $V=\emptyset$ and $V=\{1,\dots,k\}$ from the summation, i.e. the cases when either $\alpha_V(j)=1$ for all $1\le j\le k$ or $\alpha_V(j)=2$ for all $1\le j\le k$ are not considered in this sum. Formula (10.8b) can be deduced from formula~(B2) by means of a relatively simple inductive argument. In the proof of formula~(B2) we shall apply the following simple lemma. \medskip\noindent {\bf Lemma B1.} {\it Let $\xi$ and $\eta$ be two independent and identically distributed random variables taking values on a separable Banach space~$B$. Then $$ 3P\(|\xi+\eta|>\frac 23u\)\ge P(|\xi|>u)\quad \text{for all }u>0. $$ }\medskip\noindent {\it Proof of Lemma B1.}\/ {\it Let $\xi$, $\eta$ and $\zeta$ three independent, identically distributed random variables taking values in~$B$. Then $$ \align 3P\(|\xi+\eta|>\frac23 u\)&=P\(|\xi+\eta|>\frac23 u\)+ P\(|\xi+\zeta|>\frac23 u\)+P\(|-(\eta+\zeta)|>\frac23 u\)\\ &\ge P(|\xi+\eta+\xi+\zeta-\eta-\zeta|>2u)=P(|\xi|>u). \endalign $$ }\medskip To prove formula (B2) let us introduce the random variable $$ T_{n,k}(f(\ell))=\frac1{k!}\summ\Sb 1\le l_j\le n, s_j=1 \text{ or }s_j=2,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \!\!\!\!\!\!\!\!\!\!\! f_{l_1,\dots,l_k}\(\xi_{l_1}^{(s_1)},\dots,\xi_{l_k}^{(s_k)}\) = \!\!\! \sum_{V\subset\{1,\dots,k\}}\!\!\!\!\! \bar I_{n,k,V}(f(\ell)), \tag B3 $$ and observe that the random variables $I_{n,k}(f(\ell))$, $I_{n,k,\emptyset}(f(\ell))$ and $I_{n,k,\{1,\dots,k\}}(f(\ell))$ are identically distributed and the last two random variables are independent of each other. Hence Lemma~B1 yields that $$ \align P(\|I_{n,k}(f(\ell))\|>u) &\le3P\(\|I_{n,k,\emptyset}(f(\ell)) +I_{n,k,\{1,\dots,k\}}(f(\ell))\|>\frac23u\)\\ &=3P\(\left\|T_{n,k}(f(\ell))-\!\!\!\!\!\! \sum_{V\:V\subset\{1,\dots,k\},\, 1\le|V|\le k-1} I_{n,k,|V|}(f(\ell))\right\|>\frac23u\) \!\!\!\!\!\! \\ &\le P(3\cdot2^{k-1}\|T_{n,k}(f(\ell))\|>u) \tag B4 \\ &\qquad+ \!\!\!\!\!\!\!\!\! \summ_{V\:V\subset\{1,\dots,k\},\, 1\le|V|\le k-1} \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! P(3\cdot2^{k-1}\|I_{n,k,|V|}(f(\ell))\|>u). \endalign $$ To deduce relation (B2) from relation (B4) we need a good estimate on the probability $P(3\cdot2^{k-1}\|T_{n,k}(f(\ell))\|>u)$. We shall compare the distribution of $\|T_{n,k}(f(\ell))\|$ with that of $\|I_{n,k,V}(f(\ell))\|$ for an arbitrary set $V\subset\{1,\dots,k\}$ and get an estimate which is sufficient to prove relation~(B2). To carry out this program first we prove the following lemmas. \medskip\noindent {\bf Lemma B2.} {\it Let us consider a sequence of independent random variables $\e_1,\dots,\e_n$, $P(\e_l=1)=P(\e_l=-1)=\frac12$, $1\le l\le n$, which is also independent of the random variables $\xi_{1}^{(1)},\dots,\xi_n^{(1)}$ and $\xi_{1}^{(2)},\dots,\xi_n^{(2)}$ appearing in the definition of the decoupled $U$-statistics $I_{n,k,V}(f(\ell))$ defined in formula (B1). Let us define with their help the sequences of random variables $\eta_{1}^{(1)},\dots,\eta_n^{(1)}$ and $\eta_{1}^{(2)},\dots,\eta_n^{(2)}$ whose elements $(\eta_l^{(1)},\eta_l^{(2)})=(\eta_l^{(1)}(\e_l),\eta_l^{(2)}(\e_l))$, $1\le l\le n$, are given as $$ (\eta_l^{(1)}(\e_l),\eta_l^{(2)}(\e_l))=\(\frac{1+\e_l}2\xi_l^{(1)}+ \frac{1-\e_l}2\xi_l^{(2)},\frac{1-\e_l}2\xi_l^{(1)}+ \frac{1+\e_l}2\xi_l^{(2)}\), $$ i.e. let $(\eta_l^{(1)}(\e_l),\eta_l^{(2)}(\e_l))= (\xi_l^{(1)},\xi_l^{(2)})$ if $\e_l=1$, and $(\eta_l^{(1)}(\e_l),\eta_l^{(2)}(\e_l))= (\xi_l^{(2)},\xi_l^{(1)})$ if $\e_l=-1$, $1\le l\le n$. Then the joint distribution of the pair of sequences of random variables $\xi_{1}^{(1)},\dots,\xi_n^{(1)}$ and $\xi_{1}^{(2)},\dots,\xi_n^{(2)}$ agrees with that of the pair of sequences $\eta_{1}^{(1)},\dots,\eta_n^{(1)}$ and $\eta_{1}^{(2)},\dots,\eta_n^{(2)}$. Let us fix some $V\subset\{1,\dots,k\}$, and introduce the random variable $$ \bar I_{n,k,V}(f(\ell))=\frac1{k!}\summ\Sb 1\le l_j\le n,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb f_{l_1,\dots,l_k}\(\eta_{l_1}^{(\alpha_V(1))},\dots, \eta_{l_k}^{(\alpha_V(k))}\), \tag B5 $$ where similarly to formula (B1) $\alpha_V(j)=1$ if $j\in V$, and $\alpha_V(j)=2$ if $j\notin V$. Then the identity $$ \align &2^k\bar I_{n,k,V}(f(\ell)) \tag B6 \\ &\qquad=\frac1{k!}\summ\Sb 1\le l_j\le n, s_j=1 \text{ or }s_j=2,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! (1+\e^{(1)}_{l_1,s_1,V})\cdots (1+\e^{(k)}_{l_k,s_k,V}) f_{l_1,\dots,l_k}\(\xi_{l_1}^{(s_1)},\dots,\xi_{l_k}^{(s_k)}\) \endalign $$ holds, where $\e^{(j)}_{l,1,V}=\e_l$, $\e^{(j)}_{l,2,V}=-\e_l$ if $j\in V$, and $\e^{(j)}_{l,1,V}=-\e_l$, $\e^{(j)}_{l,2,V}=\e_l$ if $j\notin V$, $1\le l\le n$.} \medskip In the proof of relation (B2) we need beside Lemma~B2 another result given in Lemma~B4. Before the formulation of Lemma~B4 we present Lemma~B3 whose result will be used in its proof. \medskip\noindent {\bf Lemma B3.} {\it Let $Z$ be a random variable in a separable Banach space $B$ with expectation zero, i.e. let $E\kappa(Z)=0$ for all $\kappa\in B'$. Then $P(\|v+Z\|\ge\|x\|)\ge \inff_{\kappa\in B'} \frac{(E\kappa(Z))^2}{4E\kappa(Z)^2}$ for all $v\in B$. Here $B'$ denotes the (Banach) space of all (bounded) linear transformations on $B$ to the real line.} \medskip\noindent {\bf Lemma B4.} {\it Let us consider a sequence of independent random variables $\e_1,\dots,\e_n$, $P(\e_l=1)=P(\e_l=-1)=\frac12$, $1\le l\le n$, a polynomial of order $k$ of these random variables with some coefficients $a(l_1,\dots,l_s)$, $1\le s\le k$, $1\le l_s\le n$, from some separable Banach space $B$. Let us assume that the coefficients of this polynomial satisfy the relation $a(l_1,\dots,l_s)=0$ if $l_p=l_q$ with some $1\le p\|v\|\)\ge c_k \tag B7 $$ holds for all $v\in B$ with some constant $c_k>0$ depending only on the order $k$ of this polynomial.} \medskip\noindent {\it The proof of Lemma B2.}\/ Let us consider the conditional joint distribution of the sequences of random variables $\eta_{1}^{(1)},\dots,\eta_n^{(1)}$ and $\eta_{1}^{(2)},\dots,\eta_n^{(2)}$ under the condition that the random vector $\e_1,\dots,\e_n$ takes the value of some prescribed $\pm1$ series of length~$n$. Observe that this conditional distribution agrees with the joint distribution of the sequences $\xi_{1}^{(1)},\dots,\xi_n^{(1)}$ and $\xi_{1}^{(2)},\dots,\xi_n^{(2)}$ for all possible conditions. This fact implies the statement about the joint distribution of the sequences $\eta_l^{(1)},\eta_l^{(2)}$, $1\le l\le n$. To prove identity~(B6) let us fix a set $M\subset\{1,\dots,n\}$ and consider the case when $\e_l=1$ if $l\in M$ and $\e_l=-1$ if $l\notin M$. Observe that for all fixed sequences $1\le l_1,\dots,l_k\le n$, $l_j\neq l_{j'}$ if $j\neq j'$ $$ f_{l_1,\dots,l_k} \(\eta_{l_1}^{(\alpha_V(1))},\dots, \eta_{l_k}^{(\alpha_V(k))}\)= f_{l_1,\dots,l_k} \(\xi_{l_1}^{(\beta_{V,M}(1,l_1))},\dots, \xi_{l_k}^{(\beta_{V,M}(k,l_k))}\), $$ where $\beta_{V,M}(j,l)=1$ if $j\in V$ and $l\in M $ or $j\notin V$ and $l \notin M)$, and $\beta_{V,M}(j,l)=2$ otherwise. On the other hand, $$ \align &\summ_{s_j=1 \text{ or }s_j=2,\;j=1,\dots, k} (1+\e^{(1)}_{l_1,s_1,V})\cdots (1+\e^{(k)}_{l_k,s_k,V}) f_{l_1,\dots,l_k}\(\xi_{l_1}^{(s_1)},\dots,\xi_{l_k}^{(s_k)}\)\\ &\qquad\qquad \qquad=2^k f_{l_1,\dots,l_k} \(\xi_{l_1}^{(\beta_{V,M}(1,l_1))},\dots, \xi_{l_k}^{(\beta_{V,M}(k,l_k))}\), \endalign $$ since the product $(1+\e^{(1)}_{l_1,s_1,V})\cdots (1+\e^{(k)}_{l_k,s_k,V})$ equals either zero or $2^k$, and $\e^{(j)}_{l_j,s_j,V}=1$ if $\beta_{V,M}(j,l_j)=s_j$, and $\e^{(j)}_{l_j,s_j,V}=-1$ if $\beta_{V,M}(j,l_j)\neq s_j$. Summing up these identities for all $1\le l_1,\dots,l_k\le n$ such that $l_j\neq l_{j'}$ if $j\neq j'$ we get identity~(B6). \medskip\noindent {\it The proof of Lemma B3.}\/ Let us first observe that if $\xi$ is a real valued random variable with zero expectation, then $P(\xi>0) \ge \frac{(E|\xi|)^2}{4E\xi^2}$ since $(E|\xi|)^2 =4(E(\xi I(\{\xi>0\}))^2\le 4P(\xi>0)E\xi^2$ by the Schwarz inequality, where $I(A)$ denotes the indicator function of the set $A$. Given some $v\in B$ let us choose a linear operator $\kappa$ such that $\|\kappa\|=1$ and $\kappa(v)=\|v\|$. Such an operator exists by the Banach--Hahn theorem. Observe that $\{\oo\:\|v+Z(\oo)\|\ge \|v\|\} \supset \{\oo\: \kappa(v+Z(\oo))\ge\kappa(v)\} =\{\oo\:\kappa(Z(\oo))\ge0\}$. Beside this $E\kappa(Z)=0$. Hence we can apply the above proved inequality for $\xi=\kappa(Z)$, and it yields that $P(\|v+Z\|\ge\|v\|) \ge \frac{E\kappa(Z)^2}{4(E\kappa(Z))^2}$. Lemma~B3 is proved. \medskip\noindent {\it Proof of Lemma B4.}\/ Take the class of random polynomials $$ Y=\sum_{s=1}^k\sum \Sb 1\le l_j\le n,\; j=1,\dots, s\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb b(l_1,\dots,l_s)\e_{l_1}\cdots\e_{l_s}, $$ where $\e_l$, $1\le l\le n$, are independent random variables with $P(\e_l=1)=P(\e_l=-1)=\frac12$, and the coefficients $b(l_1,\dots,l_s)$, $1\le s\le k$, are arbitrary real numbers. It is enough to show that there exists a constant $c_k$ depending only on the order~$k$ of these polynomials such that the inequality $$ (E|Y|)^2\ge 4c_k EY^2. \tag B8 $$ holds for all of these polynomials~$Y$. Indeed, formula (B7) follows from relation~(B8) and Lemma B3 with $c_k\ge\inff_\kappa\frac{(E\kappa(Z))^2}{4E\kappa(Z)^2}$ if we apply them for the vector $v\in B$ in formula (B7) and $$ Z=\sum_{s=1}^k\sum \Sb 1\le l_j\le n,\; j=1,\dots, s\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb a(l_1,\dots,l_s)\e_{l_1}\cdots\e_{l_s}, $$ and the infimum is taken for all bounded linear operators $\kappa$ on the Banach space $B$. But this inequality follows from relation (B8). To prove relation (B8) first we compare the moments $EY^2$ and $EY^4$. Let us introduce the random variables $$ Y_s=\sum \Sb 1\le l_j\le n,\; j=1,\dots, s\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb b(l_1,\dots,l_s)\e_{l_1}\cdots\e_{l_s} \quad 1\le s\le k, $$ and observe that because of Borell's inequality (Theorem~10.2) and the uncorrelatedness of the random variables $Y_s$, $1\le s\le k$, $$ \align EY^4&=\(\sum_{s=1}^k Y_s\)^4\le k^3\sum_{s=1}^k EY_s^4\le k^3 3^{3k/2} \sum_{s=1}^k (EY_s^2)^2\\ &\le k^3 3^{3k/2}\(\sum_{s=1}^k EY_s^2\)^2=k^3 3^{3k/2}(EY^2)^2. \endalign $$ This estimate together with the H\"older inequality yield that $EY^2=E(Y^4)^{1/3}|Y|^{2/3}\le (EY^4)^{1/3}(E|Y|)^{2/3}\le k3^{k/2}(EY^2)^{1/3}(E|Y|)^{2/3}$, i.e. $EY^2\le k^{3/2}3^{3k/4}(E|Y|)^2$, and relation (B8) holds with $4c_k=k^{-3/2}3^{-3k/4}$. Lemma~B4 is proved. \medskip Let us turn back to the estimation of the probability $P(3\cdot2^{k-1}\|T_{n,k}(f)\|>u)$. Let us introduce the $\sigma$-algebra $\Cal F=\Cal B(\xi_l^{(1)},\xi_l^{(2)},\,1\le l\le n)$ generated by the random variables $\xi_l^{(1)},\xi_l^{(2)}$, $1\le l\le n$, and fix some set $V\subset\{1,\dots,k\}$. We claim that there exists some constant $c_k>0$ that the random variable $\bar I_{n,k,V}(f(\ell))$ defined in formula~(B5) satisfies the inequality $$ P\(\|2^k\bar I_{n,k,V}(f(\ell))\|>\|T_{n,k}(f(\ell))\||\Cal F\) \ge c_k \quad \text{ with probability 1.} \tag B9 $$ Indeed, formula (B6) and the independence of the random sequences $\e_{l,V}$, $\xi^{(1)}_l$ and $\xi^{(2)}_l$, $1\le l\le n$ yield that $$ \align &P\(\|2^k\bar I_{n,k,V}(f(\ell))\|>\|T_{n,k}(f(\ell))\||\Cal F\)\\ &\qquad=P_{\e_V}\biggl(\biggl\|\frac1{k!} \!\! \summ\Sb 1\le l_j\le n, s_j=1 \text{ or }s_j=2,\; j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! (1+\e^{(1)}_{l_1,s_1,V})\cdots (1+\e^{(k)}_{l_k,s_k,V}) f_{l_1,\dots,l_k}\! \(\xi_{l_1}^{(s_1)},\dots,\xi_{l_k}^{(s_k)}\)\biggr\| \\ &\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad >\|T_{n,k}(f(\ell))\|\biggr), \tag B10 \endalign $$ where $P_{\e_V}$ means that we fix the values of the random variables $\xi_l^{(1)}$, $\xi_l^{(2)}$, $1\le l\le n$ and take the probability with respect to the remaining random variables $\e^{(j)}_{l,s,V}$, $1\le j\le k$, $1\le l\le n$, and $s=1$ or $s=2$. Let us observe that the probability considered at the right-hand side of (B10) is a polynomial of order~$k$ of the random variables $\e_1,\dots,\e_n$. (The terms $\e^{(j)}_{l_j,s_j,V}$ taking part in it equal either $\e_{l_j}$ or $-\e_{l_j}$ depending on the parameters~$j$ and~$s_j$.) Beside this the constant term of this polynomial equals~$T_{n,k}(f)$. Hence this probability can be bounded by means of Lemma~B4, and this result yields relation (B9). Relation (B9) implies that $$ \align &P(\|2^k\bar I_{n,k,V}(f(\ell))\|\ge3\cdot2^{k-1} u) \\ &\qquad \ge P(\|2^k\bar I_{n,k,V}(f(\ell))\|\ge\|T_{n,k}(f(\ell))\|, \|T_{n,k}(f(\ell))\|\ge3\cdot2^{k-1} u)\\ &\qquad=\int_{\{\oo\: \|T_{n,k}(f(\ell))(\oo)\|\ge3\cdot2^{k-1} u\}} P\(\|2^k\bar I_{n,k,V}(f(\ell))\|>\|T_{n,k}(f(\ell))\||\Cal F\)\,dP\\ &\qquad \ge c_k P(\|T_{n,k}(f(\ell))\|\ge3\cdot2^{k-1} u) \endalign $$ The last inequality with the choice of any set $V\subset\{1,\dots,k\}$, $1\le |V|\le k-1$, together with relation~(B4) imply formula~(B2). To formulate the inductive hypothesis we need to prove formula (10.8b) with the help of relation~(B2) first we introduce the following quantities. Let $\Cal W=\Cal W(k)$ denote the set of all partitions of the set $\{1,\dots,k\}$. Let us fix $k$ independent copies $\xi_{1}^{(j)},\dots,\xi_n^{(j)}$, $1\le j\le k$, of the sequence of random variables $\xi_{1},\dots,\xi_n$. Given a partition $W=(V_1,\dots,V_s)\in\Cal W(k)$ let us introduce the function $s_W(j)$, $1\le j\le k$, which tells for all arguments $j$ the index of that element of the partition~$W$ which contains the point $j$, i.e. the function $s_W(j)$, $1\le j\le k$, is defined by the relation $j\in V_{s_W(j)}$. Let us define (actually generalizing the notion introduced in formula~(B1)) the notion of generalized decoupled $U$-statistics corresponding to a partition $W\in\Cal W(k)$ as $$ I_{n,k,W}(f(\ell))=\frac1{k!}\summ\Sb 1\le l_j\le n,\;j=1,\dots, k\\ l_j\neq l_{j'} \text{ if } j\neq j'\endSb f_{l_1,\dots,l_k}\(\xi_{l_1}^{(s_W(1))},\dots,\xi_{l_k}^{(s_W(k))}\) \quad \text{for all }W\in\Cal W(k). $$ Given a partition $W=(V_1,\dots,V_s)$ let us call the number $s$ of the elements of this partition the rank both of the partition $W$ and of the generalized decoupled $U$-statistic $I_{n,k,W}(f(\ell))$. Relation (10.8b) will be proved by induction with respect to the order $k$ of the $U$-statistics. This induction assumption clearly holds for $k=1$, so when we prove it for $k$ we may assume that it holds for all $k'0$ and $\delta(k,j)>0$ such that for all generalized decoupled $U$-statistics $I_{n,k,W}(f(\ell))$ of order $k$ $$ \aligned &P(\|I_{n,k,W}(f(\ell))\|>u)\le C(k,j)P\(\|\bar I_{n,k}(f(\ell))\|>\delta(k,j) u\) \\ &\qquad\text{for all }2\le j\le k \text{ if the rank of } W \text{ equals }j. \endaligned \tag B11 $$ (In relation (B11) we compare the distribution of some generalized decoupled $U$-sta\-tis\-tics with that of the decoupled $U$-statistic $\bar I_{n,k}(f(\ell))$.) We shall prove this statement by means of a backward induction with respect to the rank $j$ of the generalized decoupled $U$-statistics. Relation (B11) clearly holds for $j=k$ with $C(k,k)=1$ and $\delta(k,k)=1$. To prove it for generalized decoupled $U$-statistics of rank $2\le ju)\le \bar A(k) P\(\|I_{n,k,\bar W} (f(\ell))\|>\bar \gamma(k) u\) \tag B12 $$ with $\bar A(k)=\supp_{j\le k-1}A(j)$, $\gamma(k)=\inff_{j\le k-1}\gamma(j)$ if the rank $j$ of $W$ is such that $2\le j\le k-1$. To prove relation~(B12) let us define the $\sigma$-algebra $\Cal F$ generated by the random variables appearing in the first $s-1$ coordinates of these generalized $U$-statistics. We show that relation (10.8b) for $U$-statistics of order $k-s+1\le k-1$ yields that $P(\|I_{n,k,W}(f(\ell))\|>u|\Cal F)\le \bar A(k) P\(\|I_{n,k,\bar W}(f(\ell))\|>\bar\gamma(k) u|\Cal F\)$ with probability~1. This inequality follows from our inductive hypothesis, since the conditional probabilities we compare here are generalized $U$-statistics and generalized decoupled $U$-statistics of order $k-s+1$ we get by putting substituting the (known) first $s-1$ coordinates in the generalized $U$-statistics $I_{n,k,W}(f(\ell))$ and $I_{n,k,\bar W}(f(\ell))$. Then taking expectation at both sides of this inequality we get relation~(B12). As the rank of $\bar W$ is strictly greater than the rank of $W$ relation (B12) together with our backward inductive assumption imply relation (B11) for all $2\le j\le k$. Inequality (10.8b) is a simple consequence of relations~(B2) and~(B11). Indeed, the probability $P\(\|I_{n,k}(f(\ell))\|>u\)$ is bounded in formula~(B2) by such an expression, where some linear combination of the probabilities are considered that certain generalized decoupled $U$-statistics of order $k$ and rank~2 are larger than $uD_k^{-1}$. Each of these terms can be bounded by means of relation~(B11), and in such a way we get relation~(10.8b). We prove formula $(10.8')$ first in the simpler case when the supremum of finitely many functions is taken. Let us have $M$ functions $f_1,\dots,f_M$, and to prove relation $(10.8')$ in this case let us apply formula (10.8) with the function $f=(f_1,\dots,f_M)$ taking values in the separable Banach space $B_M$ consisting of the points $(v_1,\dots,v_M)$, $v_j\in B$, $1\le j\le M$, with the norm $\|(v_1,\dots,v_M)\|=\supp_{1\le j\le m}\|v_j\|$. The application of formula (10.8) with this choice yields formula $(10.8')$ in this case. Let us emphasize that the constants appearing in this estimate do not depend on the number $M$. Since the distribution of the random variables $\supp_{1\le s\le M} \left\| I_{n,k}(f_s)\right\|$ converge to $\supp_{1\le s<\infty} \left\| I_{n,k}(f_s)\right\|$, the distribution of the random variables $\supp_{1\le s\le M} \left\| \bar I_{n,k}(f_s)\right\|$ converge to $\supp_{1\le s<\infty} \left\|\bar I_{n,k}(f_s)\right\|$ as $M\to\infty$, we get the proof of relation $(10.8')$ in the general case by taking limit $M\to\infty$ in this relation. \beginsection Appendix C. {\it Nelson's inequality and its application} \medskip\noindent In this part of the Appendix I formulate and prove Nelson's inequality and briefly indicate how it can be applied in the proof of Theorem 8.5, i.e. in the Gaussian counterpart of Theorems~8.3 and~8.4. As the latter problem does not belong to the main subject of the work, the detailed explanation of some background results I shall apply in the proof will be omitted. In particular, I do not discuss the basic results about the properties of multiple Wiener--It\^o integrals. These results can be found for instance in my lecture note {\it Multiple Wiener--It\^o integrals}.\/ There are several equivalent formulations of Nelson's inequality. First I present its terminologically simplest form. Before its formulation let me recall that the Hermite polynomials $H_k(x)$, $k=0,1,2,\dots$, are those polynomials which constitute an orthogonal system with respect to the normal density function $\varphi(x)=\frac1{\sqrt{2\pi}}e^{-x^2/2}$. To fix their normalization, let us make the agreement that $H_k(x)$ is a polynomial of order $k$, and the coefficient of its leading term $x^k$ equals 1. \medskip\noindent {\bf Theorem C1. (Nelson's inequality).} {\it Let $(Y,\Cal Y,\nu) =(R^\infty,\Cal B^\infty,\nu^\infty)$ be the direct product of infinite many copies of the space $(R,\Cal B,\lambda_{\varphi})$, where $R$ denotes the real line, $\Cal B$ is the Borel $\sigma$-algebra on it, $\lambda_\varphi$ is the measure determined by the standard normal distribution function, i.e. the probability measure which is absolutely continuous with respect to the Lebesgue measure with density function $\varphi(y)=\frac1{\sqrt{2\pi}}e^{-y^2/2}$. Given a number $\gamma>0$ introduce the operator $\bold T_\gamma$ on $(Y,\Cal Y)$ by defining it first on polynomials by the formula $$ \align &\bold T_\gamma\(c_{l_1,j_1,\dots,l_s,j_s}\sum H_{l_1}(y_{j_1}) \cdots H_{l_s}(y_{j_s})\) \\ &\qquad =\sum \gamma^{l_1+\cdots+l_s}c_{l_1,j_1,\dots,l_s,j_s} H_{l_1}(y_{j_1})\cdots H_{l_s}(y_{j_s}), \endalign $$ where all finite sums of the above form are considered, and $H_l(\cdot)$ denotes the Hermite polynomial of order $l$. Let us extend this linear operator to general functions on the space $(Y,\Cal Y)$ in the natural way. Fix two numbers $10$ there exists some constant $C=C(K)>0$ such that $P(|Z_m^{(n)}|>x)\le Cx^K$ for all $x\ge1$ and $n=1,2,\dots$. This fact implies that relation (C4) also holds for continuous functions $f$ such that $|f(x)|\le C(1+|x|)^K)$ with some constant $C>0$ and $K>0$, where $x=(x_1,\dots,x_m)$ and $|x|$ is the length of the vector~$x$. This strengthened form of (C5) enables us to take the limit $n\to\infty$ in formula (C4) and to get relation (C3) in such a way. Let us apply formula (C3) with the choice $\xi_j=W\(\frac jm\) -W\(\frac{j-1}m\)$, $1\le j\le m$. Observe that for all indices~$l$ the inner sums at both sides of this expression are approximative sums for the Wiener--It\^o integral $\int W(\,dx_1)\cdots W(\,dx_l)$. Hence it is natural to expect that by applying the limiting procedure $m\to\infty$ in formula (C3) with the above choice of the random variables $\xi_j$ we get relation (C2). This belief is correct, only its justification requires the application of some deeper results from the theory of Wiener--It\^o integrals. We need some estimate which states that also the high moments of a Wiener--It\^o integral with a small kernel function are small. We can apply the following result. If $h$ is such a function in $[0,1]^l$ for which $\int |h(x_1,\dots,x_l)|^{2}\,dx_1\dots\,dx_l<\e$ with some $\e>0$, then also the inequality $E\left|\int h(x_1,\dots,x_l)W(\,dx_1)\dots W(\,dx_l)\right|^{2K}\le C(K,l)\e^K$ holds for all $K=1,2,\dots$ with some constant $C(K,l)$ depending only on $K$ and $l$. But the proof of this estimate demands some deeper results about Wiener--It\^o integrals. (In my lecture note about Wiener--It\^o integrals this result is proved as a consequence of the so-called diagram formula.) By applying this limiting procedure we get the proof of (C2). In such a way we have proved Proposition~C2 which, as we have shown, implies Theorem~C1. \medskip Now I formulate a version of Nelson's inequality presented in the language of Wiener--It\^o integrals. \medskip\noindent {\bf Theorem C3.} {\it Let us fix a measurable space $(X,\Cal X)$ together with a countable non-atomic measure $\mu$ on it, and let $Z_\mu$ be an orthogonal Gaussian random measure with counting measure $\mu$ on $(X,\Cal X)$. (See the definition of counting measure before the formulation of Theorem~(8.5).) For the sake of simplicity let us assume that the space $L_2(X,\Cal X,\mu)$ is separable. Let us have a sequence of measurable functions $f_k(x_1,\dots,x_k)$ on $(X^k,\Cal X^k)$ of real constant $c_k$, $k=1,2,\dots$, and also a constant $c_0$ such that $$ c_0^2+\sum_{k=1}^\infty \frac{c_k^2}{k!}\int f_k^2(x_1,\dots,x_k)\mu(\,dx_1)\dots\mu(\,dx_k)<\infty. \tag C6 $$ Then $$ \aligned &E\left|c_0+\sum_{k=1}^\infty \gamma^k \frac{c_k}{k!}\int f_k(x_1,\dots,x_k)Z_\mu(\,dx_1)\dots Z_\mu(\,dx_k)\right|^p \\ &\qquad \le \[E\left|c_0+\sum_{k=1}^\infty \frac{c_k}{k!} \int f_k(x_1,\dots,x_k) Z_\mu(\,dx_1)\dots Z_\mu(\,dx_k)\right|^q\]^{p/q}. \endaligned \tag C7 $$ if $1u)\le \frac{EZ_{\mu,k}(f)^{2M}}{u^{2M}}\le\(\frac{(2M)^k\sigma^2}{u^2}\)^M $$ if $M\ge1$. We get with the choice $M=\frac{1}{2e}\(\frac u\sigma\)^{2/k}$ that $$ P(|Z_{\mu,k}(f)|>u)\le \exp\left\{-\frac k{2e}\(\frac u\sigma\)^{2/k} \right\} \quad \text{if } u>(2e)^{k/2}\sigma. $$ By choosing a sufficiently large $A\ge1$ at the right-hand side of this inequality we get that formula (8.11) holds for all $u\ge0$. The second inequality (8.12) of Theorem~8.5 can be proved in the same way as Theorem 4.1 in the one-dimensional case. No difficulty arises during the proof. The main point is that inequality (8.11) holds for all $u>0$, hence the chaining argument applied in the proof of Theorem~4.1 supplies the proof also in this case. I omit the details. \medskip Let me finally remark that Leonhard Gross in his paper {\it Logarithmic Sobolev inequalities}\/ also gave a proof of the Nelson inequality by means of the hypercontractive inequality for Rademacher functions. He showed that the central limit theorem enables us to prove that the logarithmic Sobolev inequality holds not only for the Markov process considered in Section~11, but also for Wiener processes. This result together with the general theory he presents imply an inequality which is equivalent to our formula~(C1). \parskip=1pt plus 0.5pt \beginsection References: \item{1.)} Alexander, K. (1987) The central limit theorem for empirical processes over Vapnik--\v{C}ervonenkis classes. {\it Ann. Probab.} {\bf 15}, 178--203 \item{2.)} Arcones, M. A. and Gin\'e, E. (1993) Limit theorems for $U$-processes. {\it Ann. Probab.} {\bf 21}, 1494--1542 \item{3.)} Arcones, M. A. and Gin\'e, E. (1994) $U$-processes indexed by Vapnik--\v{C}ervonenkis classes of functions with application to asymptotics and bootstrap of $U$-statistics with estimated parameters. {\it Stoch. Proc. Appl.} {\bf 52}, 17--38 \item{4.)} Bennett, G. (1962) Probability inequality for the sum of independent random variables. {\it J. Amer. Statist. Assoc.}\/ {\bf 57}, 33-45 \item{5.)} Bonami, A. (1970) \'Etude des coefficients de Fourier des fonctions de $L^p(G)$. {\it Ann. Inst. Fourier} {\bf 20}, 335--402 \item{6.)} de la Pe\~na, V. H. and Gin\'e, E. (1999) {\it Decoupling. From dependence to independence.}\/ Springer series in statistics. Probability and its application. Springer Verlag, New York, Berlin, Heidelberg \item{7.)} de la Pe\~na, V. H. and Montgomery--Smith, S. (1995) Decoupling inequalities for the tail-probabilities of multivariate $U$-statistics. {\it Ann. Probab.}, {\bf 23}, 806--816 \item{8.)} Dudley, R. M. (1978) Central limit theorems for empirical measures. {\it Ann. Probab.}\/ {\bf 6}, 899--929 \item{9.)} Dudley, R. M. (1984) A course on empirical processes. {\it Lecture Notes in Mathemematics}\/ {\bf 1097}, 1--142 Springer Verlag, New York \item{10.)} Dudley, R. M. (1989) {\it Real Analysis and Probability.}\/ Wadsworth \& Brooks, Pacific Grove, California \item{11.)} Dudley, R. M. (1998) {\it Uniform Central Limit Theorems.}\/ Cambridge University Press, Cambridge U.K. \item{12.)} Dynkin, E. B. and Mandelbaum, A. (1983) Symmetric statistics, Poisson processes and multiple Wiener integrals. {\it Annals of Statistics\/} {\bf 11}, 739--745 \item{13.)} Gross, L. (1975) Logarithmic Sobolev inequalities. Amer. J. Math. {\bf 97}, 1061--1083 \item{14.)} Guionnet, A. and Zegarlinski, B. (2003) Lectures on Logarithmic Sobolev inequalities. {\it Lecture Notes in Mathematics} {\bf 1801} 1--134 2. Springer Verlag, New York \item{15.)} Hoeffding, W. (1963) Probability inequalities for sums of bounded random variables. {\it J. Amer. Math. Society}\/ {\bf 58}, 13--30 \item{16.)} Ledoux, M. (1996) On Talagrand deviation inequalities for product measures. {\it ESAIM: Probab. Statist.}\/ {\bf 1.} 63--87. Available at http://www.emath./fr/ps/. \item{17.)} Major, P. (1981) Multiple Wiener--It\^o integrals. {\it Lecture Notes in Mathematics\/} {\bf 849}, Springer Verlag, Berlin, Heidelberg, New York, \item{18.)} Major, P. (1988) On the tail behaviour of the distribution function of multiple stochastic integrals. {\it Probability Theory and Related Fields}, {\bf 78}, 419--435 \item{19.)} Major, P. (2005) An estimate about multiple stochastic integrals with respect to a normalized empirical measure. Submitted to {\it Studia Scientarum Mathematicarum Hungarica.} \item{20.)} Major, P. (2005) An estimate on the maximum of a nice class of stochastic integrals. Submitted to {\it Probability Theory and Related Fields}, \item{21.)} Major, P. (2005) On a multivariate version of Bernstein's inequality submitted to {\it Ann. Probab.} \item{22.)} Major, P. (2005) A multivariate generalization of Hoeffding's inequality. Submitted to {\it Ann. Probab.} \item{23.)} Major, P. (2005) On the tail behaviour of multiple random integrals and $U$-sta\-tis\-tics, on the supremum of classes of such quantities, and some related questions. (An overview work submitted to xxx) \item{24.)} Major, P. and Rejt\H{o}, L. (1988) Strong embedding of the distribution function under random censorship. {\it Annals of Statistics}, {\bf 16}, 1113--1132 \item{25.)} Major, P. and Rejt\H{o}, L. (1998) A note on nonparametric estimations. In the conference volume to the 65. birthday of Mikl\'os Cs\"org\H{o}. 759--774 \item{26.)} Massart, P. (2000) About the constants in Talagrand's concentration inequalities for empirical processes. {\it Ann. Probab.}\/ {\bf 28}, 863--884 \item{27.)} Nelson, E. (1973) The free Markov field. J. Functional Analysis {\bf 12}, 211--227 \item{28.)} Pollard, D. (1984) {\it Convergence of Stochastic Processes.}\/ Springer Verlag, New York \item{29.)} Talagrand, M. (1996) New concentration inequalities in product spaces. {\it Invent. Math.} {\bf 126}, 505--563 \item{30.)} Vapnik, V. N. (1995) {\it The Nature of Statistical Learning Theory.} Springer Verlag, New York \vfill\eject \centerline {\script Content} $$ \vbox{\halign{\hfill # \ &\vtop{\hsize=12truecm\parindent=0pt # \vskip3pt} \quad &\vtop{\hsize=0.5truecm\parindent=0pt # \vskip3pt} \cr 1. & Introduction \dotfill &\rightline{1}\cr 2. & Motivation of the investigation. Discussion of some problems \dotfill & \rightline{3}\cr 3. & Some estimates about sums of independent random variables \dotfill & \rightline{10}\cr 4. & On the supremum of a nice class of partial sums \dotfill & \rightline{15}\cr 5. & Vapnik--\v{C}ervonenkis classes and $L_2$-dense classes of functions \dotfill & \rightline{23}\cr 6. & The proof of Theorems 4.1 and 4.2 on the supremum of random sums \dotfill & \vskip5pt \rightline{26} \cr 7. & The completion of the proof of Theorem 4.1 \dotfill & \rightline{33}\cr 8. & Formulation of the main results of this work \dotfill & \rightline{40}\cr 9. & Some results about $U$-statistics \dotfill & \rightline{47}\cr 10. & The proof of Theorem 8.3 about the distribution of $U$-statistics \dotfill & \rightline{60}\cr 11. & Some useful basic results \dotfill & \rightline{69}\cr 12. & Reduction of the main result in this work \dotfill & \rightline{79}\cr 13. & The strategy of the proof for the main result of this paper \dotfill & \rightline{87}\cr 14. & A symmetrization argument \dotfill & \rightline{92}\cr 15. & The proof of the main result \dotfill &\rightline{105}\cr 16. & The improvement of some results in Section 8. \dotfill &\rightline{115}\cr 17. & An overview of the results in this work \dotfill &\rightline{121}\cr \noalign{\medskip} &Appendix A. The proof of some results about Vapnik--\v{C}ervonenkis classes \dotfill & \vskip5pt \rightline{134}\cr & Appendix B. The proof of Theorem 10.3. (A result of de le Pe\~na and Montgomery--Smith) \dotfill &\vskip5pt \rightline{135}\cr &Appendix C. Nelson's inequality and its application \dotfill &\rightline{143}\cr \noalign{\medskip} &References \dotfill & \rightline{148}\cr}} $$ \bye