**Plug-in Approach to Active Learning**

**Stanislav Minsker** SMINSKER@MATH.GATECH.EDU

*686 Cherry Street*
*School of Mathematics*

*Georgia Institute of Technology*
*Atlanta, GA 30332-0160, USA*

**Editor: Sanjoy Dasgupta**

**Abstract**

We present a new active learning algorithm based on nonparametric estimators of the regression function. Our investigation provides probabilistic bounds for the rates of convergence of the gen-eralization error achievable by proposed method over a broad class of underlying distributions. We also prove minimax lower bounds which show that the obtained rates are almost tight.

**Keywords: active learning, selective sampling, model selection, classification, confidence bands**

**1. Introduction**

Let(*S*,

*B*

)be a measurable space and let (*X*,

*Y*)

_{∈}

*S*

_{× {−}1,1

_{}}be a random couple with unknown

*distribution P. The marginal distribution of the design variable X will be denoted by* Π. Let

η(*x*):=E(*Y*_{|}*X* =*x*) *be the regression function. The goal of binary classification is to predict*
*label Y based on the observation X . Prediction is based on a classifier - a measurable *
*func-tion f : S*_{7→ {−}1,1_{}}. The quality of a classifier is measured in terms of its generalization error,
*R*(*f*) =Pr(*Y*_{6}= *f*(*X*)). In practice, the distribution P remains unknown but the learning algorithm
*has access to the training data - the i.i.d. sample*(*Xi*,*Yi*),*i*=1. . .*n from P. It often happens that the*

*cost of obtaining the training data is associated with labeling the observations Xi*while the pool of

observations itself is almost unlimited. This suggests to measure the performance of a learning
*algo-rithm in terms of its label complexity, the number of labels Yi*required to obtain a classifier with the

*desired accuracy. Active learning theory is mainly devoted to design and analysis of the algorithms*
that can take advantage of this modified framework. Most of these procedures can be characterized
*by the following property: at each step k, observation Xk* is sampled from a distribution ˆΠ*k*that

de-pends on previously obtained(*Xi*,*Yi*), *i*≤*k*−1(while passive learners obtain all available training

data at the same time). ˆΠ*k* is designed to be supported on a set where classification is difficult and

requires more labeled data to be collected. The situation when active learners outperform passive
*algorithms might occur when the so-called Tsybakov’s low noise assumption is satisfied: there exist*
*constants B*,γ>0 such that

∀*t*>0, Π(*x :*_{|}η(*x*)_{| ≤}*t*)_{≤}*Bt*γ. (1)

This assumption provides a convenient way to characterize the noise level of the problem and will play a crucial role in our investigation.

generaliza-tion error of a resulting classifier can converge to zero exponentially fast with respect to its label complexity(while the best rate for passive learning is usually polynomial with respect to the cardi-nality of the training data set). However, available algorithms that adapt to the unknown parameters

of the problem(γin Tsybakov’s low noise assumption, regularity of the decision boundary) involve

empirical risk minimization with binary loss, along with other computationally hard problems, see Balcan et al. (2008), Dasgupta et al. (2008), Hanneke (2011) and Koltchinskii (2010). On the other hand, the algorithms that can be effectively implemented, as in Castro and Nowak (2008), are not adaptive.

The majority of the previous work in the field was done under standard complexity assumptions
on the set of possible classifiers(such as polynomial growth of the covering numbers). Castro and
Nowak (2008) derived their results under the regularity conditions on the decision boundary and
the noise assumption which is slightly more restrictive then (1). Essentially, they proved that if
*the decision boundary is a graph of the H¨older smooth function g*_{∈}Σ(β,*K*,[0,1]*d*−1_{)}_{(see Section}

2 for definitions) and the noise assumption is satisfied withγ>0, then the minimax lower bound

*for the expected excess risk of the active classifier is of order C*_{·}*N*− β
(1+γ)

2β+γ(d−1) _{and the upper bound is}

*C*(*N*/*log N)*−2ββ+(1γ(d+−γ)1)_{,}_{where N is the label budget. However, the construction of the classifier that}

achieves an upper bound assumesβandγto be known.

In this paper, we consider the problem of active learning under classical nonparametric as-sumptions on the regression function - namely, we assume that it belongs to a certain H¨older class

Σ(β,*K*,[0,1]*d*_{)}_{and satisfies to the low noise condition (1) with some positive} γ_{. In this case, the}

work of Audibert and Tsybakov (2005) showed that plug-in classifiers can attain optimal rates in
*the passive learning framework, namely, that the expected excess risk of a classifier ˆg*=sign ˆηis

*bounded above by C*_{·}*N*−β

(1+γ)

2β+d _{(which is the optimal rate), where ˆ}η_{is the local polynomial estimator}

*of the regression function and N is the size of the training data set. We were able to partially extend*
this claim to the case of active learning: first, we obtain minimax lower bounds for the excess risk
of an active classifier in terms of its label complexity. Second, we propose a new algorithm that
is based on plug-in classifiers, attains almost optimal rates over a broad class of distributions and
possesses adaptivity with respect toβ,γ(within the certain range of these parameters).

The paper is organized as follows: the next section introduces remaining notations and specifies the main assumptions made throughout the paper. This is followed by a qualitative description of our learning algorithm. The second part of the work contains the statements and proofs of our main results - minimax upper and lower bounds for the excess risk.

**2. Preliminaries**

*Our active learning framework is governed by the following rules:*

*1. Observations are sampled sequentially: Xk* is sampled from the modified distribution ˆΠ*k*that

depends on(*X*1,*Y*1), . . . ,(*Xk*−1,*Yk*−1).

*2. Yk* *is sampled from the conditional distribution PY*|*X*(·|*X* =*x*). Labels are conditionally

*inde-pendent given the feature vectors Xi*,*i*≤*n.*

Usually, the distribution ˆΠ*k*is supported on a set where classification is difficult.

Given the probability measure Q *on S*_{× {−}1,1_{}}, we denote the integral with respect to this

*excess risk of f* ∈

*F*

with respect to the measureQare defined by
*R*Q(*f*):=Q

*I*

*y*6=sign f(

*x*)

*E*

Q(*f*):=

*R*Q(

*f*)−inf

*g*∈*FR*Q(*g*),

where

*I*

*A*is the indicator of event

*A*

. We will omit the subindexQwhen the underlying measure is
clear from the context. Recall that we denoted the distribution of(*X*,*Y*)*by P. The minimal possible*
*risk with respect to P is*

*R*∗= inf

*g:S*7→[−1,1]Pr(*Y* 6=*sign g(X*)),

where the infimum is taken over all measurable functions. It is well known that it is attained for any
*g such that sign g*(*x*) =signη(*x*)Π*- a.s. Given g*∈

*F*

, *A*∈

*B*

, δ>0, define
*F*

∞,*A*(

*g;*δ):={

*f*∈

*F*

: k*f*−

*g*k∞,

*A*≤δ},

where_{k}*f*_{−}*g*_{k}∞,*A*=sup
*x*∈*A*|

*f*(*x*)_{−}*g*(*x*)_{|}*. For A*_{∈}

*B*

, define the function class
*F*

_{|}

*A*:={

*f*|

*A*,

*f*∈

*F*

},
*where f*_{|}*A*(*x*):=*f*(*x*)

*I*

*A*(

*x*). From now on, we restrict our attention to the case S= [0,1]

*d. Let K*>0.

* Definition 1 We say that g :*R

*d*

_{7→}R

*belongs to*Σ(β,

*K*,[0,1]

*d*

_{)}

_{, the}_{(}

_{β}

_{,}

_{K}_{,}

_{[0}

_{,}

_{1]}

*d*

_{)}

_{- H¨older class of}*functions, if g is*_{⌊}β_{⌋}*times continuously differentiable and for all x*,*x*1∈[0,1]*dsatisfies*

|*g*(*x*1)−*Tx*(*x*1)| ≤*K*k*x*−*x*1kβ∞,
*where Tx* *is the Taylor polynomial of degree*⌊β⌋*of g at the point x.*

**Definition 2**

*P*

(β,γ)*is the class of probability distributions on*[0,1]

*d*

_{×{−}

_{1}

_{,}

_{+1}

_{}}

_{with the following}*properties:*

*1.* _{∀}*t*>0,Π(*x :*_{|}η(*x*)_{| ≤}*t*)_{≤}*Bt*γ*;*

*2.* η(*x*)_{∈}Σ(β,*K*,[0,1]*d*_{)}_{.}

We do not mention the dependence of

*P*

(β,γ)*on the fixed constants B*,

*K explicitly, but this should*not cause any uncertainty.

Finally, let us define

*P*

*∗(β,γ)and*

_{U}*P*

*U*(β,γ), the subclasses of

*P*

(β,γ), by imposing two additional
assumptions. Along with the formal descriptions of these assumptions, we shall try to provide some

motivation behind them. The first deals with the marginalΠ*. For an integer M*_{≥}1, let

*G*

*M*:=

*k*1
*M*, . . . ,

*kd*

*M*

, *ki*=1. . .*M*, *i*=1. . .*d*

be the regular grid on the unit cube[0,1]*d* * _{with mesh size M}*−1

_{. It naturally defines a partition into}

*a set of Md* _{open cubes R}

*i*, *i*=1. . .*Md* *with edges of length M*−1 and vertices in

*G*

*M*. Below, we

* Definition 3 We will say that* Π

*is*(

*u*1,

*u*2)

*-regular with respect to*{

*G*

2*m*}

*if for any m*≥

*1, any*

*element of the partition Ri*,

*i*≤2

*dmsuch that Ri*∩supp(Π)6=/0

*, we have*

*u*1·2−*dm*≤Π(*Ri*)≤*u*2·2−*dm*,
*where 0*<*u*1≤*u*2<∞*.*

**Assumption 1** Π*is*(*u*1,*u*2)*- regular.*

In particular,(*u*1,*u*2)-regularity holds for the distribution with a density p on[0,1]*d* such that 0<
*u*1≤*p*(*x*)≤*u*2<∞.

Let us mention that our definition of regularity is of rather technical nature; for most of the paper,
the reader might think ofΠas being uniform on[0,1]*d*_{( however, we need slightly more complicated}

marginal to construct the minimax lower bounds for the excess risk). It is known that estimation of
regression function in sup-norm is sensitive to the geometry of design distribution, mainly because
*the quality of estimation depends on the local amount of data at every point; conditions similar*
*to our Assumption 1 were used in the previous works where this problem appeared, for example,*
*strong density assumption in Audibert and Tsybakov (2005) and Assumption D in Ga¨ıffas (2007).*

A useful characteristic of(*u*1,*u*2)- regular distributionΠis that this property is stable with
re-spect to restrictions ofΠto certain subsets of its support. This fact fits the active learning framework
particularly well.

* Definition 4 We say that*Q

*belongs to*

*P*

*U*(β,γ)

*if*Q∈

*P*

(β,γ) *and Assumption 1 is satisfied for*

*some u*1,*u*2*.*

The second assumption is crucial in derivation of the upper bounds. The space of piecewise-constant
functions which is used to construct the estimators ofη(*x*)is defined via

*F*

*m*=

(_{2}*dm*

## ∑

*i*=1

λ*iIRi*(·): |λ*i*| ≤1, *i*=1. . .2

*dm*

)

,

where_{{}*R _{i}}*2

_{i}_{=1}

*dm*forms the dyadic partition of the unit cube. Note that

*F*

*m*can be viewed as ak · k∞

-unit ball in the linear span of first 2*dm* Haar basis functions in[0,1]*d*_{. Moreover,} _{{}

_{F}

_{F}

*m*, *m*≥1}is

a nested family, which is a desirable property for the model selection procedures. By ¯η*m*(*x*) we

*denote the L*2(Π)- projection of the regression function onto

*F*

*m*.

*We will say that the set A*_{⊂}[0,1]*d* _{approximates the decision boundary}_{{}* _{x :}*η

_{(}

_{x}_{) =}

_{0}

_{}}

_{if there}

*exists t*>0 such that

{*x :*_{|}η(*x*)_{| ≤}*t*_{}}_{Π}_{⊆}*A*_{Π}_{⊆ {}*x :*_{|}η(*x*)_{| ≤}*3t*_{}}_{Π}, (2)

*where for any set A we define A*_{Π}:=*A*_{∩}supp(Π). The most important example we have in mind

is the following: let ˆηbe some estimator ofηwith_{k}ηˆ_{−}η_{k}_{∞}_{,supp(}_{Π}_{)}_{≤}*t*,*and define the 2t - band*

aroundηby

ˆ

*F*=n*f : ˆ*η(*x*)_{−}*2t*≤ *f*(*x*)_{≤}ηˆ(*x*) +*2t* ∀*x*∈[0,1]*d*o_{.}

*Take A*=*x :* _{∃}*f*1,*f*2∈*F s.t. sign f*ˆ 1(*x*)6=*sign f*2(*x*) *, then it is easy to see that A satisfies (2).*
Modified design distributions used by our algorithm are supported on the sets with similar structure.

* Assumption 2 There exists B2*>

*0 such that for all m*≥

*1, A*∈σ(

*F*

*m*)

*satisfying (2) and such that*

*A*Π6=/0*the following holds true:*

Z

[0,1]*d*

(η_{−}η¯* _{m}*)2Π(

*dx*

_{|}

*x*

_{∈}

*A*

_{Π})

_{≥}

*B*2kη−η¯

*mk*2∞,

*A*Π.

*Appearance of Assumption 2 is motivated by the structure of our learning algorithm - namely, it is*
based on adaptive confidence bands for the regression function. Nonparametric confidence bands
is a big topic in statistical literature, and the review of this subject is not our goal. We just
men-tion that it is impossible to construct adaptive confidence bands of optimal size over the whole

S

β≤1

Σ β,*K*,[0,1]*d*_{. Low (1997); Hoffmann and Nickl (to appear) discuss the subject in details.}

*However, it is possible to construct adaptive L2* - confidence balls (see an example following

*The-orem 6.1 in Koltchinskii, 2011). For functions satisfying Assumption 2, this fact allows to obtain*
confidence bands of desired size. In particular,

(a) functions that are differentiable, with gradient being bounded away from 0 in the vicinity of decision boundary;

(b) Lipschitz continuous functions that are convex in the vicinity of decision boundary

*satisfy Assumption 2. For precise statements, see Propositions 15, 16 in Appendix A. A different*
approach to adaptive confidence bands in case of one-dimensional density estimation is presented
in Gin´e and Nickl (2010). Finally, we define

*P*

*U*∗(β,γ):

* Definition 5 We say that*Q

*belongs to*

*P*

∗
*U*(β,γ)*if* Q∈

*P*

*U*(β,γ)

*and Assumption 2 is satisfied for*

*some B*2>*0.*

**2.1 Learning Algorithm**

Now we give a brief description of the algorithm, since several definitions appear naturally in this

*context. First, let us emphasize that the marginal distribution* Π*is assumed to be known to the*

*learner. This is not a restriction, since we are not limited in the use of unlabeled data and*Πcan be
*estimated to any desired accuracy. Our construction is based on so-called plug-in classifiers of the*
form ˆ*f*(_{·}) =sign ˆη(_{·}), where ˆηis a piecewise-constant estimator of the regression function. As we
have already mentioned above, it was shown in Audibert and Tsybakov (2005) that in the passive

*learning framework plug-in classifiers attain optimal rate for the excess risk of order N*−β(

1+γ)

2β+d _{, with}

ˆ

ηbeing the local polynomial estimator.

Our active learning algorithm iteratively improves the classifier by constructing shrinking

*con-fidence bands for the regression function. On every step k, the piecewise-constant estimator ˆ*η*k* is

obtained via the model selection procedure which allows adaptation to the unknown smoothness(for
H¨older exponent_{≤}1). The estimator is further used to construct a confidence band ˆ

*F*

*k*forη(

*x*). The

*active set associated with ˆ*

*F*

*k*is defined as

ˆ

*Ak*=*A*(

*F*

ˆ*k*):=

n

*x*_{∈}supp(Π): _{∃}*f*1,*f*2∈

*F*

ˆ*k*,

*sign f*1(

*x*)6=

*sign f*2(

*x*)

o

.

*requested only for observations X*∈* _{A}*ˆ

_{k}_{, forcing the labeled data to concentrate in the domain where}higher precision is needed. This allows one to obtain a tighter confidence band for the regression function restricted to the active set. Since ˆ

*Ak*approaches the decision boundary, its size is controlled

by the low noise assumption. The algorithm does not require a priori knowledge of the noise and
regularity parameters, being adaptive forγ>0,β_{≤}1. Further details are given in Section 3.2.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1.5

−1 −0.5 0 0.5 1 1.5

**confidence**
**band**

η**(x)− regression**
**function**

**estimator **
**of **η**(x)**

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1.5

−1 −0.5 0 0.5 1 1.5

**confidence**
**band**

**ACTIVE**
**SET**

Figure 1: Active Learning Algorithm

**2.2 Comparison Inequalities**

Before proceeding with the main results, let us recall the well-known connections between the
binary risk and the_{k · k}_{∞},_{k · kL2}_{(}_{Π}_{)}- norm risks:

**Proposition 6 Under the low noise assumption,**

*RP*(*f*)−*R*∗≤*D*1k(*f*−η)

*I*

{*sign f*6=signη}k1+

_{∞}γ; (3)

*RP*(*f*)−*R*∗≤*D*2k(*f*−η)

*I*

{*sign f*6=signη}k

2(1+γ)

2+γ

*L2*(Π); (4)

*RP*(*f*)−*R*∗≥*D*3Π(sign f 6=signη)

1+γ

γ _{.} _{(5)}

**Proof For (3) and (4), see Audibert and Tsybakov (2005), Lemmas 5.1 and 5.2 respectively, and**
for (5)—Koltchinskii (2011), Lemma 5.2.

**3. Main Results**

The question we address below is: what are the best possible rates that can be achieved by active algorithms in our framework and how these rates can be attained.

**3.1 Minimax Lower Bounds For the Excess Risk**

*The goal of this section is to prove that for P*_{∈}

*P*

(β,γ), no active learner can output a classifier with
*expected excess risk converging to zero faster than N*− β

(1+γ)

2β+d−βγ_{. Our result builds upon the minimax}

**Remark The theorem below is proved for a smaller class**

*P*

∗
*U*(β,γ), which implies the result for

*P*

(β,γ).
* Theorem 7 Let*β,γ,

*d be such that*βγ

_{≤}

*d. Then there exists C*>

*0 such that for all n large enough*

*and for any active classifier ˆfn*(

*x*)

*we have*

sup

*P*∈*P*∗

*U*(β,γ)

E*RP*(*f*ˆ*n*)−*R*∗≥*CN*−
β(1+γ)

2β+d−βγ_{.}

* Proof We proceed by constructing the appropriate family of classifiers f*σ(

*x*) =signησ(

*x*), in a way similar to Theorem 3.5 in Audibert and Tsybakov (2005), and then apply Theorem 2.5 from Tsybakov (2009). We present it below for reader’s convenience.

* Theorem 8 Let* Σ

*be a class of models, d :*Σ

_{×}Σ

_{7→}R

*- the pseudometric and*

*Pf*,

*f*∈Σ

*- a*

*collection of probability measures associated with*Σ*. Assume there exists a subset*_{{}*f*0, . . . ,*fM*}*of*Σ

*such that*

*1. d*(*fi*,*fj*)≥*2s*>*0 for all 0*≤*i*< *j*≤*M;*

*2. Pfj* ≪*Pf0* *for every 1*≤ *j*≤*M;*

*3.* * _{M}*1

*M*

∑

*j*=1

KL(*Pfj*,*Pf0*)≤α*log M*, 0<α<
1
8*.*
*Then*

inf ˆ

*f*

sup

*f*_{∈}Σ

*Pf* d(*f*ˆ,*f*)≥*s*

≥ √

*M*

1+√*M* 1−2α−

s

2α
*log M*

!

,

*where the infimum is taken over all possible estimators of f based on a sample from Pf* *and KL*(·,·)

*is the Kullback-Leibler divergence.*

*Going back to the proof, let q*=2*l*,*l*_{≥}1 and

*Gq*:=

*2k1*_{−}1

*2q* , . . . ,

*2kd*−1

*2q*

,*ki*=1. . .*q*,*i*=1. . .*d*

be the grid on[0,1]*d _{. For x}*

_{∈}

_{[0}

_{,}

_{1]}

*d*

_{, let}

*nq*(*x*) =argmin k*x*−*xk*k2*: xk*∈*Gq* .

*If nq*(*x*)is not unique, we choose a representative with the smallest k · k2 norm. The unit cube is
*partitioned with respect to Gq* *as follows: x*1,*x*2 *belong to the same subset if nq*(*x*1) =*nq*(*x*2). Let

′_{≻}′ _{be some order on the elements of G}_{q}_{such that x}_{≻}_{y implies}_{k}_{x}_{k}_{2}_{≥ k}_{y}_{k}_{2. Assume that the}

elements of the partition are enumerated with respect to the order of their centers induced by′_{≻}′:

[0,1]*d*_{=} *q*

*d*

S

*i*=1

*Ri*. Fix 1≤*m*≤*qd*and let

*S :*=

*m*

[

Note that the partition is ordered in such a way that there always exists 1≤*k*≤*q*√*d with*

*B*+

0,*k*
*q*

⊆*S*_{⊆}*B*+ 0,*k*+3

√

*d*
*q*

!

, (6)

*where B*+(0,*R*):=*x*∈R*d*+: k*x*k2≤*R* . In other words, (6) means that that the difference between
*the radii of inscribed and circumscribed spherical sectors of S is of order C(d*)*q*−1.

*Let v*>*r*1>*r*2be three integers satisfying

2−*v*<2−*r1*<2−*r1*√*d*<2−*r2*√*d*<2−1. (7)

*Define u(x*):R_{7→}R_{+}by

*u*(*x*):=

R_{∞}

*x* *U*(*t*)*dt*

1/2_{R}
2−*v*

*U*(*t*)*dt*

, (8)

where

*U*(*t*):=

(

exp

− 1

(1/2−*x*)(*x*_{−}2−*v*_{)}

, *x*∈(2−*v*_{,}1
2)

0 else.

*Note that u(x*) *is an infinitely differentiable function such that u(x*) =1, *x*_{∈}[0,2−*v*]*and u(x*) =
0, *x*_{≥}1_{2}*. Finally, for x*_{∈}R*d*let

Φ(*x*):=*Cu*(_{k}*x*_{k}2),

*where C :=CL*,βis chosen such thatΦ∈Σ(β,*L*,R*d*).
*Let rS*:=inf{*r*>*0 : B*+(0,*r*)⊇*S*}and

*A*0:=

(

[

*i*

*Ri: Ri*∩*B*+

0,*rS*+*q*−
βγ

*d*

= /0

)

.

Note that

*rS*≤*c*

*m*1/*d*

*q* , (9)

since Vol(*S*) =*mq*−*d*.

Define

*H*

*m*={

*P*σ:σ∈ {−1,1}

*m*}to be the hypercube of probability distributions on[0,1]

*d*× {−1,+1

_{}}. The marginal distributionΠ

*of X is independent of*σ

*: define its density p by*

*p*(*x*) =

2*d(r1*−1)

2*d(r1*−*r2*)_{−}_{1}, *x*∈*B*∞

*z*,2−*r2*

*q*

\*B*_{∞}*z*,2−*r1*

*q*

, *z*_{∈}*Gq*∩*S*,

*c*0, *x*∈*A*0,

0 else.

*where B*∞(*z*,*r*):=_{{}*x :* _{k}*x*_{−}*z*_{k}∞≤*r*_{}}*, c0*:= 1_{Vol(}−*mq _{A0}*−

_{)}

*d*(note thatΠ(

*Ri*) =

*q*−

*d*∀

*i*≤

*m) and r*1,

*r*2are

defined in (7). In particular,Π*satisfies Assumption 1 since it is supported on the union of dyadic*

cubes and has bounded above and below on supp(Π)density. Let

Ψ(*x*):=*u*1/2_{−}*q*βγ*d*dist2(* _{x}*,

*+(0,*

_{B}*))*

_{r}_{S}

Figure 2: Geometry of the support

*where u(*_{·})is defined in (8) and dist2(*x*,*A*):=inf{k*x*−*y*k2,*y*∈*A*}.
Finally, the regression functionη_{σ}(*x*) =E_{P}_{σ}(*Y*_{|}*X*=*x*)is defined via

ησ(*x*):=

( _{σ}

*iq*−βΦ(*q*[*x*−*nq*(*x*)]), *x*∈*Ri*, 1≤*i*≤*m*

1

*CL*,β

√

*d*dist2(*x*,*B*+(0,*rS*))

*d*

γ_{·}Ψ_{(}_{x}_{)}_{,} _{x}_{∈}_{[0}_{,}_{1]}*d*_{\}_{S}_{.}

The graph ofη_{σ} *is a surface consisting of small ”bumps” spread around S and tending away from*

0 monotonically with respect to dist2(·,*B*+(0,*rS*))on[0,1]*d*\*S. Clearly,*ησ(*x*)satisfies smoothness

requirement,1*since for x*_{∈}[0,1]*d*

dist2(*x*,*B*+(0,*rS*)) = (k*x*k2−*rS*)∨0.

Let’s check that it also satisfies the low noise condition. Since_{|}η_{σ}_{| ≥}*Cq*−βon the support ofΠ, it
*is enough to consider t*=*Czq*−β*for z*>1:

Π(_{|}ησ(*x*)_{| ≤}*Czq*−β)_{≤}*mq*−*d*+Πdist2(*x*,*B*+(0,*rS*))≤*Cz*γ/*dq*−
βγ

*d*

≤

≤*mq*−*d*+*C*2

*rS*+*Cz*γ/*dq*−
βγ

*d*

*d*
≤

≤*mq*−*d*+*C*3*mq*−*d*+*C*4*z*γ*q*−βγ≤

≤*Ct*bγ,

*if mq*−*d*=*O*(*q*−βγ). Here, the first inequality follows from consideringησ*on S and A0* separately,
and second inequality follows from (9) and direct computation of the sphere volume.

Finally,η_{σ}*satisfies Assumption 2 with some B*2:=*B*2(*q*)since on supp(Π)
0<*c*1(*q*)_{≤ k}∇ησ(*x*)_{k}2≤*c*2(*q*)<∞.

The next step in the proof is to choose the subset of

*H*

which is “well-separated”: this can be done
due to the following fact (see Tsybakov, 2009, Lemma 2.9):

* Proposition 9 (Gilbert-Varshamov) For m*≥

*8, there exists*

{σ0, . . . ,σ*M*} ⊂ {−1,1}*m*

*such that*σ0={1,1, . . . ,1}*,*ρ(σ*i*,σ*j*)≥ *m*_{8} ∀0≤*i*<*k*≤*M and M*≥2*m*/8 *where*ρ*stands for the*

*Hamming distance.*

Let

*H*

′_{:=}

_{{}

_{P}_{σ}

0, . . . ,*P*σ*M*} be chosen such that{σ0, . . . ,σ*M*} satisfies the proposition above. Next,
following the proof of Theorems 1 and 3 in Castro and Nowak (2008), we note that_{∀}σ_{∈}

*H*

′,σ_{6}=σ0

KL(*P*_{σ},*N*k*P*σ0,*N*)≤*8N max*

*x*∈[0,1](ησ(*x*)−ησ0(*x*))
2

≤*32C _{L}*2

_{,}

_{β}

*Nq*−2β, (10)

*where P*_{σ},*N* is the joint distribution of (*Xi*,*Yi*)*Ni*=1 under hypothesis that the distribution of couple
(*X*,*Y*)*is P*σ. Let us briefly sketch the derivation of (10); see also the proof of Theorem 1 in Castro
and Nowak (2008). Denote

¯

*Xk*:= (*X*1, . . . ,*Xk*),

¯

*Yk* := (*Y*1, . . . ,*Yk*).

*Then dP*_{σ},*N*admits the following factorization:

*dP*_{σ},*N*(*X*¯*N*,*Y*¯*N*) =
*N*

## ∏

*i*=1

*P*_{σ}(*Y _{i|}Xi*)

*dP*(

*Xi|X*¯

*i*−1,

*Y*¯

*i*−1),

*where dP(Xi*|*X*¯*i*_{−}1,*Y*¯*i*_{−}1) does not depend onσ but only on the active learning algorithm. As a

consequence,

KL(*P*_{σ},*NkP*σ0,*N*) =E*P*σ,*N*log

*dP*_{σ},*N*(*X*¯*N*,*Y*¯*N*)

*dP*σ0,*N*(*X*¯*n*,*Y*¯*N*)

=E_{P}_{σ,}* _{N}*log ∏

*N*

*i*=1*P*σ(*Yi|Xi*)

∏*N*

*i*=1*P*σ0(*Yi*|*Xi*)

=

=

*N*

## ∑

*i*=1

E_{P}_{σ,}_{N}

E_{P}_{σ}

log *P*σ(*Yi|Xi*)
*P*_{σ}_{0}(*Y _{i|}Xi*)|

*Xi*

≤

≤*N max*

*x*∈[0,1]*d*

E_{P}_{σ}

log *P*σ(*Y*1|*X*1)
*P*σ0(*Y*1|*X*1)

|*X*1=*x*

≤

≤*8N max*

*x*∈[0,1]*d*(ησ(*x*)−ησ0(*x*))
2_{,}

where the last inequality follows from Lemma 1 (Castro and Nowak, 2008). Also, note that we have
max_{x}_{∈}_{[0,1]}*d* *in our bounds rather than the average over x that would appear in the passive learning*
framework.

*It remains to choose q*,*m in appropriate way: set q*=_{⌊}*C*1*N*

1

2β+d−βγ_{⌋}_{and m}_{=}_{⌊}_{C}_{2}_{q}d_{−}βγ_{⌋}_{where}

*C*1, *C*2 *are such that qd* ≥*m*≥*1 and 32C*2_{L}_{,}_{β}*Nq*−2β< _{64}*m* *which is possible for N big enough. In*
*particular, mq*−*d*=*O*(*q*−βγ). Together with the bound (10), this gives

1
*M*_{σ}

## ∑

∈*H*′

KL(*P*σk*P*_{σ}0)≤*32C _{u}*2

*Nq*−2β<

*m*
82 =

1 8log|

*H*

so that conditions of Theorem 8 are satisfied. Setting

*f*σ(*x*):=signησ(*x*),

we finally have_{∀}σ16=σ2∈

*H*

′
*d*(*f*_{σ}_{1},*f*_{σ}_{2}):=Π(signη_{σ}_{1}(*x*)_{6}=signη_{σ}_{2}(*x*))_{≥} *m*

*8qd* ≥*C*4*N*

−_{2β}+dβγ−βγ_{,}

where the lower bound just follows by construction of our hypotheses. Since under the low noise
*assumption RP*(*f*ˆ*n*)−*R*∗≥*c*Π(*f*ˆ*n*6=signη)

1+γ

γ _{(see (5)), we conclude by Theorem 8 that}

inf ˆ

*fN*
sup

*P*∈*P*∗

*U*(β,γ)
Pr

*RP*(*f*ˆ*n*)−*R*∗≥*C*4*N*−

β(1+γ)

2β+d−βγ

≥

≥inf

ˆ

*fN*
sup

*P*∈*P*∗

*U*(β,γ)
Pr

Π(*f*ˆ*n*(*x*)6=signη*P*(*x*))≥

*C*4

2 *N*

−2β+dβγ−βγ

≥τ>0.

**3.2 Upper Bounds For the Excess Risk**

Below, we present a new active learning algorithm which is computationally tractable, adaptive
with respect toβ,γ( in a certain range of these parameters) and can be applied in the nonparametric
setting. We show that the classifier constructed by the algorithm attains the rates of Theorem 7, up
to polylogarithmic factor, if 0<β_{≤}1 andβγ_{≤}*d (the last condition covers the most interesting case*
when the regression function hits or crosses the decision boundary in the interior of the support ofΠ;
for detailed statement about the connection between the behavior of the regression function near the

decision boundary with parametersβ, γ, see Proposition 3.4 in Audibert and Tsybakov, 2005). The

problem of adaptation to higher order of smoothness (β>1) is still awaiting its complete solution; we address these questions below in our final remarks.

For the purpose of this section, the regularity assumption reads as follows: there exists 0<β_{≤}1
such that_{∀}*x*1,*x*2∈[0,1]*d*

|η(*x*1)−η(*x*2)| ≤*B*1k*x*1−*x*2kβ_{∞}. (11)

Since we want to be able to construct non-asymptotic confidence bands, some estimates on the size
*of constants in (11) and Assumption 2 are needed. Below, we will additionally assume that*

*B*1≤*log N*,
*B*2≥log−1*N*,

*where N is the label budget. This can be replaced by any known bounds on B*1,*B*2.

*Let A*_{∈}σ(

*F*

*m*)

*with A*Π:=

*A*∩supp(Π)6=/0. Define

ˆ

*and dm*:=dim

*F*

*m|A*Π. Next, we introduce a simple estimator of the regression function on the set

*A*Π*. Given the resolution level m and an iid sample*(*Xi*,*Yi*),*i*≤*N with Xi*∼Πˆ*A*, let

ˆ

η*m*,*A*(*x*):=

## ∑

*i:Ri*∩

*A*Π6=/0

∑*N*

*j*=1*Yj*

*I*

*Ri*(

*Xj*)

*N*·_{Π}ˆ_{A}_{(}_{R}_{i}_{)}

*I*

*Ri*(

*x*). (12)

Since we assumed that the marginal Π is known, the estimator is well-defined. The following

proposition provides the information about concentration of ˆη*m*around its mean:

* Proposition 10 For all t*>

*0,*

Pr max

*x*∈*A*Π|

ˆ

η*m*,*A*(*x*)−η¯*m*(*x*)| ≥*t*

s

2*dm*Π_{(}_{A}_{)}

*u*1*N*

!

≤

≤*2dm*exp −

*t*2
2(1+_{3}*t*p2*dm*Π_{(}_{A}_{)}_{/}_{u}

1*N*)

!

,

**Proof This is a straightforward application of the Bernstein’s inequality to the random variables**

*Si _{N}*:=

*N*

## ∑

*j*=1

*Yj*

*I*

*Ri*(

*Xj*),

*i*∈ {

*i : Ri*∩

*A*Π6= /0},

and the union bound: indeed, note thatE(*Y*

*I*

*Ri*(

*Xj*)) 2

_{=}Π

_{ˆ}

*A*(*Ri*), so that

Pr_{}*Si _{N}*

_{−}

*N*

Z

*Ri*

η*d ˆ*Π*A*

≥*tN ˆ*Π*A*(*Ri*)

≤2 exp

−*N ˆ*Π*A*(*Ri*)*t*

2
2+*2t*/3

,

and the rest follows by simple algebra using that ˆΠ*A*(*Ri*)≥_{2}*dmu1*_{Π}_{(}_{A}_{)} by the(*u*1,*u*2)-regularity ofΠ.

Given a sequence of hypotheses classes

*G*

*m*,

*m*≥1, define the index set

*J*

(*N*):=

*m*∈N: 1≤dim

*G*

*m*≤

*N*
log2*N*

(13)

*- the set of possible “resolution levels” of an estimator based on N classified observations(an upper*
bound corresponds to the fact that we want the estimator to be consistent). When talking about
model selection procedures below, we will implicitly assume that the model index is chosen from

the corresponding set

*J*

. The role of*G*

*m*will be played by

*F*

*m|Afor appropriately chosen set A. We*

are now ready to present the active learning algorithm followed by its detailed analysis(see Table 1).

**Remark Note that on every iteration, Algorithm 1a uses the whole sample to select the **
resolu-tion level ˆ*mk* and to build the estimator ˆη*k*. While being suitable for practical implementation, this

is not convenient for theoretical analysis. We will prove the upper bounds for a slighly modified
*version: namely, on every iteration k labeled data is divided into two subsamples Sk*,1 *and Sk*,2 of
approximately equal size,_{|}*Sk*,1| ≃ |*Sk*,2| ≃1_{2}*Nk*·Π(*A*ˆ*k*)*. Then S1,k* is used to select the resolution

**Algorithm 1a**

* input label budget N; confidence*α;
ˆ

*m*0=0,

*F*

ˆ0:=*F*

*m0*ˆ ,ηˆ0≡0;

*LB :*=*N;* *// label budget*

*N*0:=2⌊log2

√

*N*⌋_{;}

*s*(*k*)(*m*,*N*,α):=*s*(*m*,*N*,α):=*m*(log N+log_{α}1);
*k :*=0;

**while LB**_{≥}**0 do**
*k :*=*k*+1;
*Nk*:=*2Nk*−1;

ˆ
*Ak*:=

n

*x*_{∈}[0,1]*d*_{:} _{∃}_{f}

1,*f*2∈

*F*

ˆ*k*−1,sign(

*f*1(

*x*))6=sign(

*f*2(

*x*))

o

;

**if ˆ***Ak*∩supp(Π) =/0* or LB*<⌊

*Nk*·Π(

*A*ˆ

*k*)⌋

**then**

**break; output ˆ***g :*=sign ˆη*k*_{−}1
**else**

* for i*=1. . .

_{⌊}

*Nk*·Π(

*A*ˆ

*k*)⌋

**sample i.i.d** *X _{i}*(

*k*),

*Y*(

_{i}*k*)

*(*

**with X**_{i}*k*)

_{∼}Πˆ

*k*:=Π(

*dx*|

*x*∈

*A*ˆ

*k*);

**end for;**

*LB :*=*LB*_{− ⌊}*Nk*·Π(*A*ˆ*k*)⌋;

ˆ

*Pk*:= 1

⌊*Nk*·Π(*A*ˆ*k*)⌋∑* _{i}* δ

*Xi*(k),

*Y*(k)

*i*

*// ”active” empirical measure*

ˆ

*mk*:=argmin*m*≥*m*ˆ*k*−1

h

inf*f*∈*FmP*ˆ*k*(*Y*−*f*(*X*))
2_{+}_{K}

12
*dm*_{Π}_{(}_{A}_{ˆ}

*k*)+*s*(*m*−*m*ˆ*k*−1,*N*,α)

⌊*Nk*·Π(*A*ˆ*k*)⌋

i

ˆ

η*k*:=ηˆ_{m}_{ˆ}_{k}_{,}* _{A}*ˆ

_{k}*// see (12)*

δ*k*:=*D*˜·log*2 N*_{α}

q

2*d ˆ _{mk}*

*Nk* ;
ˆ

*F*

*k*:=

n

*f* _{∈}

*F*

*m*ˆ

*k*

*: f*|

*A*ˆ

*k*∈

*F*

∞,*A*ˆ

*k*(ηˆ

*k*;δ

*k*),

*f*|[0,1]

*d*\

*A*ˆ

*k*≡ηˆ

*k*−1|[0,1]

*d*\

*A*ˆ

*k*

o

;

**end;**

Table 1: Active Learning Algorithm

**As a first step towards the analysis of Algorithm 1b, let us prove the useful fact about the general**
model selection scheme. Given an iid sample(*Xi*,*Yi*),*i*≤*N, set sm*=*m*(*s*+log log2*N*),*m*≥1 and

ˆ

*m :*=*m*ˆ(*s*) =argmin_{m}_{∈}_{J}_{(}_{N}_{)}

inf

*f*∈*Fm*

*PN*(*Y*−*f*(*X*))2+*K*1

2*dm*+*sm*

*N*

,

¯

*m :*=min

*m*≥1 : inf

*f*∈*Fm*

E(*f*(*X*)_{−}η(*X*))2_{≤}* _{K}*
2

2*dm*

*N*

.

**Theorem 11 There exist an absolute constant K1**big enough such that, with probability_{≥}1_{−}*e*−*s,*

ˆ
*m*_{≤}*m*¯.

**Proof See Appendix B.**

* Corollary 12 Suppose*η(

*x*)

_{∈}Σ(β,

*L*,[0,1]

*d*

_{)}

_{. Then, with probability}_{≥}

_{1}

_{−}

*−*

_{e}*s*

_{,}2*m*ˆ _{≤}*C*1·*N*

1 2β+d

**Proof By definition of ¯***m, we have*

¯

*m*_{≤}1+max

*m : inf*

*f*∈*Fm*

E(*f*(*X*)_{−}η(*X*))2>*K*2
2*dm*

*N*

≤

≤1+max

*m : L*22−2β*m*>*K*2
2*dm*

*N*

,

and the claim follows.

With this bound in hand, we are ready to formulate and prove the main result of this section:

**Theorem 13 Suppose that P**_{∈}

*P*

∗
*U*(β,γ) *with B*1≤*log N*, *B*2≥log−1*N and* βγ≤*d. Then, with*
*probability*_{≥}1_{−}3α*, the classifier ˆ g returned by Algorithm 1b with label budget N satisfies*

*RP*(*g*ˆ)−*R*∗≤Const·*N*−
β(1+γ)

2β+d−βγ_{log}*pN*

α,

*where p*_{≤}_{2}2_{β}βγ_{+}(1+_{d}_{−}_{βγ}γ) *and B*1, *B*2*are the constants from (11) and Assumption 2.*
**Remarks**

1. Note that when βγ> *d*

3*, N*

−_{2β}β+d(1+−γ)βγ * _{is a fast rate, that is, faster than N}*−1

2; at the same time,

*the passive learning rate N*−

β(1+γ)

2β+d _{is guaranteed to be fast only when}βγ_{>}*d*

2, see Audibert and Tsybakov (2005).

2. For ˆα_{≃}*N*− β
(1+γ)

2β+d−βγ _{Algorithm 1b returns a classifier ˆ}_{g}_{α}_{ˆ} _{that satisfies}

E*RP*(*g*ˆαˆ)−*R*∗≤Const·*N*−

β(1+γ)

2β+d−βγ_{log}*p _{N}*

_{.}

This is a direct corollary of Theorem 13 and the inequality

E_{|}*Z*_{| ≤}*t*+_{k}*Z*_{k}∞Pr(_{|}*Z*_{| ≥}*t*).

**Proof Our main goal is to construct high probability bounds for the size of the active sets defined**

**by Algorithm 1b. In turn, these bounds depend on the size of the confidence bands for**η(*x*), and

*the previous result(Theorem 11) is used to obtain the required estimates. Suppose L is the number*

*of steps performed by the algorithm before termination; clearly, L*_{≤}*N.*

*Let N _{k}*act:=

_{⌊}

*Nk*·Π(

*A*ˆ

*k*)⌋

*be the number of labels requested on k-th step of the algorithm: this*

choice guarantees that the ”density” of labeled examples doubles on every step.

Claim: the following bound for the size of the active set holds uniformly for all 2_{≤}*k*_{≤}*L with*

probability at least 1_{−}2α:

Π(* _{A}*ˆ

_{k}_{)}

_{≤}

*−2ββγ+d*

_{CN}*k*

log*N*_{α}

2γ

It is not hard to finish the proof assuming (14) is true: indeed, it implies that the number of labels
*requested on step k satisfies*

*N _{k}*act=

_{⌊}

*Nk*Π(

*A*ˆ

*k*)⌋ ≤

*C*·

*N*2β+d−βγ

2β+d

*k*

log*N*

α

2γ

with probability_{≥}1_{−}2α. Since∑

*k*

*N _{k}*act

_{≤}

*N, one easily deduces that on the last iteration L we have*

*NL*≥*c*

*N*
log2γ(*N*/α)

2β+d

2β+d−βγ

(15)

To obtain the risk bound of the theorem from here, we apply2inequality (3) from Proposition 6:

*RP*(*g*ˆ)−*R*∗≤*D*1k(ηˆ*L*−η)·

*I*

{sign ˆη*L*=6 signη}k1+

_{∞}γ. (16)

It remains to estimatekηˆ*L*−ηk_{∞}_{,}* _{A}*ˆ

*: we will show below while proving (14) that*

_{L}kηˆ*L*−ηk_{∞}_{,}*A*ˆ*L*≤*C*·*N*

−2ββ+d

*L* log2

*N*

α.

Together with (15) and (16), it implies the final result.

To finish the proof, it remains to establish (14). Recall that ¯η*kstands for the L2(*Π)- projection

ofηonto

*F*

*m*ˆ

*k. An important role in the argument is played by the bound on the L2(*Πˆ

*k*)- norm of the “bias”(η¯

_{k}_{−}η): together with Assumption 2, it allows to estimate

_{k}η¯

_{k}_{−}η

_{k}

_{∞}

_{,}

_{A}_{ˆ}

*k*. The required

bound follows from the following oracle inequality: there exists an event

*B*

of probability_{≥}1

_{−}α

such that on this event for every 1_{≤}*k*_{≤}*L*

kη¯_{k}_{−}η_{k}2

*L2*(_{Π}ˆ_{k}_{)}≤ inf

*m*≥*m*ˆ*k*−1

"

inf

*f*_{∈}*Fm*k

*f*_{−}η_{k}2_{L2}_{(}_{Π}_{ˆ}

*k*)+ (17)

+*K*1

2*dm*Π(* _{A}*ˆ

_{k}_{) + (}

_{m}_{−}

_{m}_{ˆ}

_{k}_{−}

_{1}

_{)}

_{log(}

_{N}_{/}

_{α}

_{)}

*Nk*Π(

*A*ˆ

*k*)

#

.

It general form, this inequality is given by Theorem 6.1 in Koltchinskii (2011) and provides the
estimate for _{k}ηˆ*k*−ηkL2_{(}_{Π}ˆ_{k}_{)}, so it automatically implies the weaker bound for the bias term only.
*To deduce (17), we use the mentioned general inequality L times(once for every iteration) and the*
union bound. The quantity 2*dm*Π(*A*ˆ*k*) in (17) plays the role of the dimension, which is justified

*below. Let k*_{≥}*1 be fixed. For m*_{≥}*m*ˆ*k*−1, consider hypothesis classes

*F*

*m*|

*ˆ*

_{A}*:=*

_{k}n

*f*

*I*

*ˆ*

_{A}*,*

_{k}*f*∈

*F*

*m*

o

.

*An obvious but important fact is that for P*_{∈}

*P*

*U*(β,γ), the dimension of

*F*

*m|A*ˆ

*k*

*is bounded by u*

−1
1 ·
2*m*Π(* _{A}*ˆ

_{k}_{): indeed,}

Π(* _{A}*ˆ

_{k}_{) =}

## ∑

*j:Rj*∩*A*ˆ*k*6=/0

Π(*Rj*)≥*u*12−*dm*·#

*j : Rj*∩*A*ˆ*k*6=/0 ,

hence

dim

*F*

*m*|

*ˆ*

_{A}*=#*

_{k}*j : Rj*∩

*A*ˆ

*k*=6 /0 ≤

*u*−

_{1}1·2

*m*Π(

*A*ˆ

*k*). (18)

Theorem 11 applies conditionally on n*X _{i}*(

*j*)o

*Nj*

*i*=1, *j*≤*k*−*1 with sample of size N*

act

*k* *and s*=

log(*N*/α): to apply the theorem, note that, by definition of ˆ*Ak, it is independent of Xi*(*k*),*i*=1. . .*Nk*act.

Arguing as in Corollary 12 and using (18), we conclude that the following inequality holds with
probability_{≥}1_{−}α_{N}*for every fixed k:*

2*m*ˆ*k*_{≤}_{C}_{·}_{N}

1 2β+d

*k* . (19)

Let

*E*

1be an event of probability≥1−α*such that on this event bound (19) holds for every step k,*

*k*_{≤}*L and let*

*E*

2be an event of probability≥1−αon which inequalities (17) are satisfied. Suppose
that event

*E*

1∩*E*

2 *occurs and let k*0 be a fixed arbitrary integer 2≤

*k*0≤

*L*+1. It is enough to

assume that ˆ*Ak0*−1is nonempty(otherwise, the bound trivially holds), so that it contains at least one
cube with sidelength 2−*m*ˆ*k0*−2 _{and}

Π(*A*ˆ*k0*−1)≥*u*12−*d ˆmk0*−1≥*cN*−

*d*

2β+d

*k0* . (20)

*Consider inequality (17) with k*=*k*0−1 and 2*m*≃*N*

1 2β+d

*k0*−1. By (20), we have

kη¯*k0*_{−}1−ηk* _{L2}*2

_{(}

_{Π}ˆ

_{k0}_{−}

_{1}

_{)}≤

*CN*

−2β2β+d

*k0*−1 log
2*N*

α. (21)

For convenience and brevity, denoteΩ:=supp(Π). Now Assumption 2 comes into play: it implies,

together with (21) that

*CN*−

β 2β+d

*k0*−1 log
*N*

α ≥ kη¯*k0*−1−ηkL2_{(}_{Π}ˆ

*k0*−1)≥*B*2kη¯*k0*−1−ηk∞,Ω∩*A*ˆ* _{k0}*−1. (22)

To bound

kηˆ*k0*_{−}1(*x*)−η¯*k0*_{−}1(*x*)k_{∞}_{,}_{Ω}_{∩}* _{A}*ˆ

*k0*−1

we apply Proposition 10. Recall that ˆ*mk0*−1*depends only on the subsample Sk0*−1,1*but not on Sk0*−1,2.
Let

*T*

*k*:=

n

*X _{i}*(

*j*),

*Y*(

_{i}*j*)o

*N*

act

*j*

*i*=1, *j*≤*k*−1

∪*Sk*,1

be the random vector that defines ˆ*Ak*and resolution level ˆ*mk. Note that for any x,*

E(ηˆ*k0*−1(*x*)|

*T*

*k0*−1)a.s.=η¯

*m*ˆ

_{k0}_{−}1(

*x*).

Proposition 10 thus implies

Pr max

*x*∈Ω∩*A*ˆ_{k0}_{−}1

|ηˆ*k0*−1(*x*)−η¯*m*ˆ_{k0}_{−}1(*x*)| ≥*Kt*

s

2*d ˆmk0*−1

*Nk0*_{−}1

*T*

*k0*−1

!

≤

≤*N exp*

−*t*2

2(1+*t*

3*C*3)

*Choosing t*=*c log*(*N*/α)and taking expectation, the inequality(now unconditional) becomes

Pr

max

*x*∈Ω∩* _{A}*ˆ

*k0*−1

|ηˆ*m*ˆ_{k0}_{−}1(*x*)−η¯*m*ˆ* _{k0}*−1(

*x*)| ≤

*K*

s

2*d ˆmk0*−1_{log}2_{(}_{N}_{/}α_{)}

*Nk0*−1

_{≥}1_{−}α. (23)

Let

*E*

3be the event on which (23) holds true. Combined, the estimates (19),(22) and (23) imply that
on

*E*

1∩*E*

2∩*E*

3
kη−ηˆ*k0*−1k_{∞}_{,}_{Ω}_{∩}* _{A}*ˆ

*k0*−1 ≤ kη−η¯*k0*−1k∞,Ω∩*A*ˆ* _{k0}*−1+kη¯

*k0*−1−ηˆ

*k0*−1k∞,Ω∩

*A*ˆ

*−1*

_{k0}≤ *C*

*B*2
*N*−

β 2β+d

*k0*−1 log
*N*

α+*K*

s

2*d ˆmk0*−1_{log}2_{(}_{N}_{/}α_{)}

*Nk0*−1 ≤

(24)

≤(*K*+*C*)_{·}*N*−

β 2β+d

*k0*−1 log
2*N*

α,

*where we used the assumption B2*≥log−1*N. Now the width of the confidence band is defined via*

δ*k*:=2(*K*+*C*)·*N*

−2ββ+d

*k0*−1 log
2*N*

α (25)

(in particular, ˜* D from Algorithm 1a is equal to 2*(

*K*+

*C*)). With the bound (24) available, it is

straightforward to finish the proof of the claim. Indeed, by (25) and the definition of the active set,
*the necessary condition for x*_{∈}Ω_{∩}*A*ˆ*k0* is

|η(*x*)_{| ≤}3(*K*+*C*)_{·}*N*−

β 2β+d

*k0*−1 log
2*N*

α,

so that

Π(*A*ˆ*k0*) =Π(Ω∩*A*ˆ*k0*)≤Π

|η(*x*)_{| ≤}3(*K*+*C*)_{·}*N*−

β 2β+d

*k0*−1 log
2*N*

α

≤

≤* _{BN}*˜ −

βγ 2β+d

*k0*−1 log
2γ*N*

α.

by the low noise assumption. This completes the proof of the claim since Pr(

*E*

1∩*E*

2∩*E*

3)≥
1_{−}3α.

We conclude this section by discussing running time of the active learning algorithm. Assume that
*the algorithm has access to the sampling subroutine that, given A*_{⊂}[0,1]*d*_{with}_{Π}_{(}_{A}_{)}_{>}_{0, generates}

i.i.d.(*Xi*,*Yi*)*with Xi*∼Π(*dx*|*x*∈*A*).

**Proposition 14 The running time of Algorithm 1a(1b) with label budget N is**

*O*

(*dN log*2

*N*).

**Remark In view of Theorem 13, the running time required to output a classifier ˆ***g such that RP*(*g*ˆ)−

*R*∗_{≤}εwith probability_{≥}1_{−}αis

*O*

1

ε

2β+d−βγ

β(1+γ) poly

log 1

εα

* Proof We will use the notations of Theorem 13. Let N_{k}*act be the number of labels requested by

*the algorithm on step k. The resolution level ˆmk*is always chosen such that ˆ

*Ak*is partitioned into

*at most N _{k}*act dyadic cubes, see (13). This means that the estimator ˆη

*k*

*takes at most Nk*act distinct

*values. The key observation is that for any k, the active set ˆAk*+1is always represented as the union
*of a finite number(at most N _{k}*act

*) of dyadic cubes: to determine if a cube Rj*⊂

*A*ˆ

*k*+1, it is enough to

*take a point x*_{∈}*Rj*and compare sign(ηˆ*k*(*x*)−δ*k*)with sign(ηˆ*k*(*x*) +δ*k*): R*j*∈*A*ˆ*k*+1only if the signs
are different(so that the confidence band crosses zero level). This can be done in

*O*

(*N*act

*k* )steps.

Next, resolution level ˆ*mk*can be found in

*O*

(*N*actlog2

_{k}*N*)steps: there are at most log2

*Nk*actmodels

*to consider; for each m, inff*∈*FmP*ˆ*k*(*Y*−*f*(*X*))

2_{is found explicitly and is achieved for the }
piecewise-constant ˆ*f*(*x*) = ∑*iY*

(k)
*i* *IR j*(*Xi*(k))

∑*iIR j*(*Xi*(k))

, *x*_{∈}*Rj*.Sorting of the data required for this computation is done in

*O*

(*dN*act

_{k}*log N)steps for each m, so the whole k-th iteration running time is*

*O*

(*dN*actlog2

_{k}*N*). Since

∑

*k*

*N _{k}*act

_{≤}

*N, the result follows.*

**4. Conclusion and Open Problems**

We have shown that active learning can significantly improve the quality of a classifier over the passive algorithm for a large class of underlying distributions. Presented method achieves fast rates of convergence for the excess risk, moreover, it is adaptive(in the certain range of smoothness and

noise parameters) and involves minimization only with respect to quadratic loss(rather than the 0_{−}1

loss).

The natural question related to our results is:

• Can we implement adaptive smooth estimators in the learning algorithm to extend our results

beyond the caseβ_{≤}1?

The answer to this second question is so far an open problem. Our conjecture is that the correct

*rate of convergence for the excess risk is, up to logarithmic factors, N*− β

(1+γ)

2β+d−γ(β∧1)_{,}_{which coincides}

with presented results forβ_{≤}1. This rate can be derived from an argument similar to the proof of

*Theorem 13 under the assumption that on every step k one could construct an estimator ˆ*η*k* with

kη−ηˆ_{k}_{k}_{∞}_{,}* _{A}*ˆ

*.*

_{k}*N*

−2ββ+d

*k* .

At the same time, the active set associated to ˆη*k* should maintain some structure which is suitable

for the iterative nature of the algorithm. Transforming these ideas into a rigorous proof is a goal of our future work.

**Acknowledgments**

I want to express my sincere gratitude to my Ph.D. advisor, Dr. Vladimir Koltchinskii, for his support and numerous helpful discussions.

I am grateful to the anonymous reviewers for carefully reading the manuscript. Their insightful and wise suggestions helped to improve the quality of presentation and results.

**Appendix A. Functions Satisfying Assumption 2**

In the propositions below, we will assume for simplicity that the marginal distributionΠis

*abso-lutely continuous with respect to Lebesgue measure with density p(x*)such that

0<*p*1≤*p*(*x*)≤*p*2<∞*for all x*∈[0,1]*d*.
*Given t*_{∈}(0,1], define A*t* :={*x :* |η(*x*)| ≤*t*}.

* Proposition 15 Suppose*η

*is Lipschitz continuous with Lipschitz constant S. Assume also that for*

*some t*

_{∗}>

*0 we have*

*(a)* Π *At*_{∗}/3

>*0;*

*(b)* η*is twice differentiable for all x*_{∈}*At*_{∗}*;*

*(c) infx*∈*At*_{∗}k∇η(*x*)k1≥*s*>0;
*(d) sup _{x}*

_{∈}

_{A}_{t}∗k*D*

2η_{(}_{x}_{)}_{k ≤}_{C}_{<}∞_{where}_{k · k}_{is the operator norm.}*Then*η*satisfies Assumption 2.*

* Proof By intermediate value theorem, for any cube Ri*, 1≤

*i*≤2

*dm*

*there exists x0*∈

*Ri*such that

¯

η*m*(*x*) =η(*x*0),*x*∈*Ri*. This implies

|η(*x*)_{−}η¯* _{m}*(

*x*)

_{|}=

_{|}η(

*x*)

_{−}η(

*x*0)|=|∇η(ξ)·(

*x*−

*x*0)| ≤

≤ k∇η(ξ)_{k}1k*x*−*x*0k∞≤*S*·2−*m*.
*On the other hand, if Ri*⊂*At*∗ then

|η(*x*)_{−}η¯* _{m}*(

*x*)

_{|}=

_{|}η(

*x*)

_{−}η(

*x*0)|= =

_{|}∇η(

*x*0)·(

*x*−

*x*0) +

1
2[*D*

2_{η}_{(}_{ξ}_{)](}_{x}_{−}_{x}

0)·(*x*−*x*0)| ≥

≥ |∇η(*x*0)_{·}(*x*_{−}*x*0)_{| −}1
2sup_{ξ} k*D*

2_{η}_{(}_{ξ}_{)}

kmax

*x*_{∈}*Ri*k

*x*_{−}*x*0k22≥ (26)

≥ |∇η(*x*0)·(*x*−*x*0)| −*C*12−*2m*.
Note that a strictly positive continuous function

*h*(*y*,*u*) =

Z

[0,1]*d*

(*u*_{·}(*x*_{−}*y*))2*dx*

*achieves its minimal value h*_{∗}>0 on a compact set[0,1]*d*_{×}_{u}_{∈}_{R}*d*_{:}_{k}_{u}_{k}

1=1 . This implies(using

(26) and the inequality(*a*_{−}*b*)2_{≥} *a*2

2 −*b*
2_{)}

Π−1_{(}_{R}

*i*)

Z

*Ri*

(η(*x*)_{−}η¯*m*(*x*))2*p*(*x*)*dx*≥

≥1_{2}(*p*22*dm*)−1

Z

*Ri*

(∇η(*x*0)·(*x*−*x*0))2*p*1*dx*−*C*122−*4m*≥

≥1

2
*p*1
*p*2k

*Now take a set A*∈σ(

*F*

*m*),

*m*≥

*m*0

*from Assumption 2. There are 2 possibilities: either A*⊂

*At*∗ or

*A*_{⊃}*At*∗/3. In the first case the computation above implies

Z

[0,1]*d*

(η_{−}η¯*m*)2Π(*dx*|*x*∈*A*)≥*c*22−*2m*=
*c*2
*S*2*S*

2_{2}−*2m*_{≥}

≥* _{S}c*22kη−η¯

*m*k 2 ∞,

*A*.

If the second case occurs, note that, since*x : 0*<_{|}η(*x*)_{|}<*t*∗

3 has nonempty interior, it must

*con-tain a dyadic cube R*_{∗}with edge length 2−*m*∗_{. Then for any m}_{≥}_{max(}_{m}_{0}_{,}_{m}

∗) Z

[0,1]*d*

(η_{−}η¯* _{m}*)2Π(

*dx*

_{|}

*x*

_{∈}

*A*)

_{≥}

≥Π−1(*A*)

Z

*R*∗

(η_{−}η¯*m*)2Π(*dx*)≥

*c*2

42

−*2m*_{Π}_{(}_{R}

∗)≥

≥*c _{S}*2

_{2}Π(

*R*

_{∗})

_{k}η

_{−}η¯

*2*

_{mk}_{∞}

_{,}

*and the claim follows.*

_{A}The next proposition describes conditions which allow functions to have vanishing gradient on decision boundary but requires convexity and regular behaviour of the gradient.

Everywhere below,∇ηdenotes the subgradient of a convex functionη.

For 0<*t*1<*t*2*, define G(t*1,*t*2):=
sup
*x*∈* _{At2}*\

*k*

_{At1}∇η(*x*)k1

inf

*x*∈* _{At2}*\

*k∇η(*

_{At1}*x*)k1

. In case when∇η(*x*)is not unique, we choose

*a representative that makes G(t*1,*t*2)as small as possible.

* Proposition 16 Suppose*η(

*x*)

*is Lipschitz continuous with Lipschitz constant S. Moreover, assume*

*that there exists t*

_{∗}>

*0 and q :*(0,∞)

_{7→}(0,∞)

*such that At*

_{∗}⊂ (0,1)

*dand*

*(a) b*1*t*γ≤Π(*At*)≤*b*2*t*γ∀*t*<*t*∗*;*

*(b) For all 0*<*t*1<*t*2≤*t*∗,*G*(*t*1,*t*2)≤*q*

*t2*
*t1*

*;*

*(c) Restriction of*η*to any convex subset of At*∗ *is convex.*

*Then*η*satisfies Assumption 2.*

* Remark The statement remains valid if we replace*η

*by*

_{|}η

_{|}

*in (c).*

**Proof Assume that for some t**_{≤}*t*_{∗}*and k*>0

*R*_{⊂}*At*\*At*/*k*

is a dyadic cube with edge length 2−*m* *and let x0* be such that ¯η*m*(*x*) =η(*x*0), *x*∈*R. Note that*η

*is convex on R due to (c). Using the subgradient inequality*η(*x*)_{−}η(*x*0)_{≥}∇η(*x*0)_{·}(*x*_{−}*x*0), we
obtain

Z

*R*

(η(*x*)_{−}η(*x*0))2*d*Π(*x*)_{≥}

Z

*R*

(η(*x*)_{−}η(*x*0))2

*I*

{∇η(*x*0)

_{·}(

*x*

_{−}

*x*0)

_{≥}0

_{}}

*d*Π(

*x*)

≥

Z

*R*

*The next step is to show that under our assumptions x0*can be chosen such that

dist∞(*x*0,∂*R*)≥ν2−*m*, (28)

where ν=ν(*k*) *is independent of m. In this case any part of R cut by a hyperplane through x0*

*contains half of a ball B(x*0,*r*0)*of radius r*0=ν(*k*)2−*m* and the last integral in (27) can be further
bounded below to get

Z

*R*

(η(*x*)_{−}η(*x*0))2*d*Π(*x*)≥
1
2

Z

*B*(*x0*,*r0*)

(∇η(*x*0)·(*x*−*x*0))2*p*1*dx*≥

≥*c*(*k*)_{k}∇η(*x*0)_{k}2_{1}2−*2m*2−*dm*. (29)

*It remains to show (28). Assume that for all y such that*η(*y*) =η(*x*0)we have
dist_{∞}(*y*,∂*R*)_{≤}δ2−*m*

for someδ>0. This implies that the boundary of the convex set

{*x*_{∈}*R :*η(*x*)_{≤}η(*x*0)_{}}

*is contained in R*_{δ} := _{{}*x*_{∈}*R : dist*∞(*x*,∂*R*)_{≤}δ2−*m*_{}}. There are two possibilities: either

{*x*_{∈}*R :*η(*x*)_{≤}η(*x*0)} ⊇*R*\*R*δor{*x*∈*R :*η(*x*)≤η(*x*0)} ⊂*R*δ.

We consider the first case only(the proof in the second case is similar). First, note that by (b) for all
*x*_{∈}*R*_{δ}_{k}∇η(*x*)_{k}1≤*q*(*k*)k∇η(*x*0)k1and

η(*x*)_{≤}η(*x*0) +_{k}∇η(*x*)_{k}1δ2−*m*≤

≤η(*x*0) +*q*(*k*)_{k}∇η(*x*0)_{k}1δ2−*m*. (30)

*Let xcbe the center of the cube R and u - the unit vector in direction*∇η(*xc*). Observe that

η(*xc*+ (1−3δ)2−*mu*)−η(*xc*)≥∇η(*xc*)·(1−3δ)2−*mu*=

= (1_{−}3δ)2−*m*_{k}∇η(*xc*)k2.
*On the other hand, xc*+ (1−3δ)2−*mu*∈*R*\*R*δand

η(*xc*+ (1−3δ)2−*mu*)≤η(*x*0),
henceη(*xc*)≤η(*x*0)−*c*(1−3δ)2−*m*k∇η(*xc*)k1. Consequently, for all

*x*∈*B*(*xc*,δ):=

*x :*k*x*−*xck*∞≤1
2*c2*

−*m*_{(1}_{−}_{3}_{δ}_{)}

we have

η(*x*)_{≤}η(*xc*) +k∇η(*xc*)k1k*x*−*xc*k∞≤
≤η(*x*0)_{−}1

2*c2*

−*m*_{(1}