2 7 1 0
2 ndInternaitona lConferenceonAritifcialI ntelilgenceandEngineeirngAppilcaitons(AIEA2017)
8 7 9 : N B S
I -1-60595- 54 -1 8
t
r
o
p
p
u
S
t
s
u
b
o
R
l
e
v
o
N
a
f
o
s
is
y
l
a
n
A
e
c
n
a
m
r
o
f
r
e
P
m
h
ti
r
o
g
l
A
n
o
is
s
e
r
g
e
R
r
o
t
c
e
V
U X O A H C , V L N A U
Y da n BINZHONG
T C A R T S B A
r o f e g n e ll a h c e g u h a s i n o it a b r u t r e p l a r d e h y l o p h ti w a t a d l a t n e m i r e p x e e h T
. g n it s a c e r o f a t a d d n a l e d o m n o i s s e r g e
r This paper aims to study on regression
t r o p p u s t s u b o r e h t f o e c n a m r o f r e
p vector regression for processing inpu tdata wtih
n o it a b r u t r e p l a r d e h y l o
p which was proposed by the author of this paper .First ,the c u d o r t n i s a w d o h t e m n o i s s e r g e r r o t c e v t r o p p u s t s u b o r e h t f o l e d o m l a c it a m e h t a
m ed
y l a n a s a w n o it a b r u t r e p l a r d e h y l o p e h t f o e r u t a e f e h t d n
a zed .Second ,the robus t
l e n r e k n a i s s u a G d e s u y l e d i w t s o m e h t d n a d e t s il s a w w o l f m h ti r o g l a n o i s s e r g e r
. t n e s e r p s a w n o it c n u
f Third ,two numerica lexperiments including ilnear regression a
e n il n o n d n
a r regression were given to prove effecitveness of the robus tregression .
n o it a b r u t r e p l a r d e h y l o p h ti w a t a d t u p n i g n i s s e c o r p r o f d o h t e m
KEYWORDS
. n o it a b r u t r e P l a r d e h y l o P , n o i s s e r g e R r o t c e V t r o p p u S , t s u b o R
N O I T C U D O R T N I
d e i f it n a u q e h t y r e v o c s i d o t r e d r o n
I relaitonshipbetweenthe targe tvalue and the
n o i s s e r g e r e m o s g n i s u y b d e h s il b a t s e e r e w s l e d o m l a c it a m e h t a m e h t , s r o t c a f g n it c e f f a
n a i s e y a b , g n i n r a e l e e r t n o i s i c e d , n o i s s e r g e r s e r a u q s t s a e l : e l p m a x e r o f , s m h ti r o g l a
t r a , ) M V S ( e n i h c a m r o t c e v t r o p p u s , d o h t e
m ificia lneura lnetworks and deep learning
e h t t a h t s i s e h t o p y h e h t n o d e s a b e r a s d o h t e m g n i n r a e l e v o b a e h t l l A . ] 1 [ . c t e
. e s i c e r p s i a t a d l a n o it a v r e s b o r o a t a d l a t n e m i r e p x
e In fact ,i tis inevtiable tha tthe
v h ti w d e t a n i m a t n o c y ll a u s u e r a a t a d l a t n e m i r e p x
e arious errors and noise .In other
e h t , r e v e w o H . d e b r u t r e p e r a s l e d o m l a c it a m e h t a m e h t f o a t a d t u p n i e h t l l a , s d r o w
a t a d t u p n i d e b r u t r e p e h t f o e c n e u l f n
i ont heexistenceandopitmaltiyoft hesoluitont o
. ] 2 [ e m it g n o l r o f d e r o n g i s a w l e d o m n o i s s e r g e r e h t
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _______ _
n a u
Y vL , Schoo lofMechanica lEngineeirng, X’ianUniverstiy ofScienceandTechnology,
n a ’i
X 710054,China;l [email protected]
o a h
C X , u SchoolofMechanica lEngineeirng,X’ianUniverstiy ofScienceandTechnology,
n a ’i
X 710054,China;[email protected]
g n o h Z n i
B , Schoo l of Mechanica l Engineeirng, X’ian Universtiy of Science and
n a ’i X , y g o l o n h c e
In 1973 ,Soyster [3] proved tha tthe ilnear programming wtih perturbed data in t
e s x e v n o
c is solvable firslty .Latter ,the rapid developmen tof robus topitmizaiton ,
s e t r o C a n n i r o C y b d e t o m o r p s a w y g o l o n h c e
t JohnPlatt ,NelloCrisitaniniandJieLiu
4 [ y lt n e d n e p e d n
i -7] .In2009and2013,t herobus tsuppor tvectorclassificaitonmethod n
e v n i s a
w tedbyNaiyangDeng[8] .Subsequenlty,t herobus tsuppor tvectorregression .
e l c it r a s i h t f o r o h t u a e h t y b d e s o p o r p s a w d o h t e
m Andi thasbeenproveneffecitvei n
[ n o it a b r u t r e p m r o f i r e h p s h ti w a t a d t u p n i g n i s s e c o r
p 9 .]
r o r r e s u o i r a v d n a s u o r e m u n e h t o t e u
D s of the observed data ,the types and the
n o m m o c t s o m e h T . t n e r e f f i d e r a a t a d d e v r e s b o e h t f o t e s n o it a b r u t r e p e h t f o s e r u t c u r t s
. n o it a b r u t r e p l a r d e h y l o p d n a n o it a b r u t r e p m r o f i r e h p s e d u l c n i s t e s n o it a b r u t r e p
e p l a r d e h y l o p e h t , e n o r e m r o f e h t h ti w g n i r a p m o
C rturbaitonismorecompilcatedand
o t s m i a r e p a p s i h T . d e v l o s e r d n a d e t a g it s e v n i n e e b t o n s a h m e l b o r p n o i s s e r g e r s ti
s e s y l a n
a theregressionperformanceoft herobus tsuppor tvectorregressionmode.l
W O L F M H T I R O G L A D N A L E D O M L A C I T A M E H T A M
c e V t r o p p u S t s u b o
R to rRegres isonModel
Suppor tvectormachinereailzest hefuncitonofdataregressionandforecasitngby .
e n a l p r e p y h l a m it p o n a g n i z il it
u Therefore ,searching the opitma lhyperplane is the
s s e r g e r r o t c e v t r o p p u s e h t f o l e d o m l a c it a m e h t a m e h t g n i d li u b f o y e
k ion for
. n o it a b r u t r e p l a r d e h y l o p h ti w a t a d d e v r e s b o g n i s s e c o r
p For this purpose ,the training
e h t s e b i r c s e d h c i h w m e l b o r p n o it a z i m it p o e h t d n a n o it a b r u t r e p l a r d e h y l o p e h t ,t e s a t a d
. y a w g n i w o ll o f e h t n i n e v i g e r a e n a l p r e p y h l a m it p o e h t g n i h c r a e s f o s s e c o r p
: w o ll o f s a d e b i r c s e d s i T t e s a t a d g n i n i a r t e h T
� 1�
The inpu tdata in equaiton (1) is determined by perturbaiton center , e
d u ti l p m a n o it a b r u t r e
p and int hefollowingmanne r:
( 2)
: e r e h W
:independen tvariables ,wecalledasinpu tdata; :dependen tvariable ,wecalledasoutpu tdata;
c n o it a b r u t r e p
: enterofi npu tdata; ; a t a d t u p n i f o e d u ti l p m a n o it a b r u t r e p :
n a
: matrix,t he1-normofthecolumnvector isl essthanorequalt o d
n a
, ai s givenrea lnumber; : numberoft rainingdata;
a t a d t u p n i f o y ti l a n o i s n e m i d :
n .
h ti w a t a d d e r u s a e m g n i s s e c o r p r o f l e d o m n o i s s e r g e r r o t c e v t r o p p u s t s u b o r e h T
a s i n o it a b r u t r e p l a r d e h y l o
( 3)
Themathemaitca lmode l(3)derivedfromt heclassic�-suppor tvectorregression .
l e d o
m Theconvexquadraitcprogrammingcouldbeconvertedt oasecond-ordercone n
o m m o c e h t e r a h s y e h t d n a , g n i m m a r g o r
p soluiton. The common soluiton of the
d n o c e s e h t d n a g n i m m a r g o r p c it a r d a u q x e v n o
c - dor er cone programming is derived
d n o c e s e h t f o g n i m m a r g o r p l a u d e h t f o n o it u l o s e h t m o r
f -order cone programming .
[ e r u t a r e ti l n i n e e s s i s s e c o r p n o it a v i r e d r a li m i s t u o b a n o it a m r o f n i s li a t e d e r o
M 9 .]
w o l F m h ti r o g l A n o is s e r g e R r o t c e V t r o p p u S t s u b o R
The specific flow of the robus tsuppor tvector regression algortihm is ilsted as .
1 e r u g i
f By using the kerne lfunciton ,the ilnear robus tregression algortihm for r a e n il n o n e h t o t d e d n e t x e y li s a e s i n o it a b r u t r e p l a r d e h y l o p h ti w a t a d t u p n i g n i s s e c o r p
. a e r a
Kernelf unc it on
Theperformanceoft herobus tregressionmethodi sdirecltyrelatedtot heseleciton f
o appropriate kerne lfunciton .The common kerne lfuncitons include the following s
e n
o .
n o it c n u f l e n r e k r a e n i L ) 1 (
( 4)
n o it c n u f l e n r e k n a i s s u a G ) 2 (
( 5)
n o it c n u f l e n r e k l a i m o n y l o P ) 3 (
( 6)
n o it c n u f l e n r e k d i o m g i S ) 4 (
( 7)
) 4 ( s n o it a u q e n
I -(7) ,the variables are given system parameters of the .
s n o it c n u f l e n r e
k Gaussian kerne lis the mos tcommon and widely used funciton fo r .
s t r a p o w t o t n i a t a d l a t n e m i r e p x e g n i t r a p e D
a t a d g n i n i a r
T Inequation（1）
l e d o m l a c i t a m e h t a m e h t g n i t c u r t s n o
C (3 )of
e n a l p r e p y h l a m i t p o e h t
d n o c e s a o t l e d o m e h t g n i t r e v n o
C -order
g n i m m a r g o r p e n o c
f o m e l b o r p l a u d e h t f o n o i t u l o s g n i r i u q c A
d n o c e s e h
t -orderconeprogramming
l a c i t a m e h t a m e h t f o n o i t u l o s g n i n i a t b O
l e d o
m (3)
o i t c n u f g n i t s a c e r o f e h t g n i d li u
B n
l e d o m g n i t s a c e r o f l a n i f e h t g n i t t u p t u O
Te
st
da
ta
r o r r e d e t c i d e r p f I
d l o h s e r h t n a h t s s e l
e u l a
v ?
s e Y o
N
Te
st
da
ta
be
co
m
es
t
ra
ini
ng
d
at
a
F gi u 1. re RobustRegressionAlgortihmFlow.
S T N E M I R E P X E L A C I R E M U N
s t n e m i r e p x E l a c i r e m u N r a e n i L
. 1 e l p m a x
E The ilnearfunciton was chosenastheobjecitve funciton ,and e
l b a i r a v t u p n i e h
t is a2dimensions vector .Accordingto the formula (2) ,theinpu t s
a d e t o n e d s i t e s
. The center
a s i a t a d t u p n i d e b r u t s i d f
o matrix .The disturbing quanttiy
a s
i matrix .The value of isa random number in [-1 1] ,and the f
o e u l a
v is a random number in [0 1] , . The variable is calculated a
l u m r o f e h t m o r
f .
Basedonthetradiitona lstandard ilnearsuppor tvectorregression methodand the m
n o i s s e r g e r r o t c e v t r o p p u s r a e n il t s u b o
. d e t c u r t s n o c e r e w s l e d o m n o i s s e r g e r r a e n il t n e r e f f i d o w t , r e p a p s i h
t In order to
s t n i o p a t a d 7 6 , y c a r u c c a n o it c i d e r p r i e h t e r a p m o
c ta
, 1 . 5 , 5 [ =
x 5.3,5.5,5.7,5.9,⋯,15.1,15.3,15.5,15.7,15.9,16] were selected as tes t data . Afterrunningt hemodelsonacomputer,t heerrorsbetweenforecastedandrea lresutls
. 1 e r u g i f n i t n e s e r p e r e w
s t n e m i r e p x E l a c i r e m u N r a e n il n o N
. 2 e l p m a x
E Nonilnearfunciton
s a n e s o h c s a w b
o e h
t jecitvefunciton ,andtheinpu tvariable isa2dimensionsvector .Thevalue y
ti t n a u q g n i b r u t s i d e h t , a t a d t u p n i d e b r u t s i d e h t f o r e t n e c e h t f
o andt hevariable
l p m a x e s a y a w e m a s e h t n i d e n i a t b o e r e
w e1 .
Basedonthetradiitonalstandardnonilnear suppor tvectorregressionmethod and n o it c e s n i d e s o p o r p s a w h c i h w d o h t e m n o i s s e r g e r r o t c e v t r o p p u s r a e n il n o n t s u b o r e h t
. d e t c u r t s n o c e r e w s l e d o m n o i s s e r g e r r a e n il n o n t n e r e f f i d o w t , r e p a p s i h t n i 2 . 4 .
2 I n
ordert ocomparet heirpredicitonaccuracy ,67pointsat ,
9 . 5 , 7 . 5 , 5 . 5 , 3 . 5 , 1 . 5 , 5 [ =
x ⋯,15.1,15.3,15.5,15.7,15.9,16] werechosenast es tdata .After e r e w s tl u s e r l a e r d n a d e t s a c e r o f n e e w t e b s r o r r e e h t , r e t u p m o c a n o s l e d o m e h t g n i n n u r
. 2 e r u g i f n i t n e s e r p
4 5 6 7 8 9 10 11 12 13 14 15 16
4 . 0
-3 . 0
-2 . 0
-1 . 0
-0 1 . 0
2 . 0
3 . 0
X a t a D t u p n I
E r
orr
E
n
oit
ci
d
er
P
R V S t s u b o R r a e n i L f o r o r r E n o it i c i d e r P
R V S d r a d n a t S r a e n i L f o r o r r E n o it i c i d e r P
9 6 4 1 . 0 = r o r r E n o it i d e r P f o e u l a V e t u l o s b A n a e M
8 5 2 0 . 0 = r o r r E n o it c i d e r P f o e u l a V e t u l o s b A n a e M
g i
F u 2. re ComparisonChar tofPredicitonErrorofTwoLinearSVRAlgortihm . s
4 5 6 7 8 9 10 11 12 13 14 15 16
5 1
-0 1
-5
-0 5 0 1
5 1
X a t a D t u p n I
E r
orr
E
n
oit
ci
d
er
P
R V S t s u b o R r a e n il n o N f o r o r r E n o it c i d e r P
R V S d r a d n a t S r a e n il n o N f o r o r r E n o it c i d e r P
5 5 5 3 . 0 = r o r r E n o it c i d e r P f o e u l a V e t u l o s b A n a e M
5 8 6 4 . 3 = r o r r E n o it c i d e r P f o e u l a V e t u l o s b A n a e M
g i
n o is s u c si D
In figure 1 ,the blue stars are the prediciton errors of the ilnear robus tSVR .
m h ti r o g l
a The red circles are the prediciton errors of the ilnear standard SVR i
r o g l
a thm .Obviously ,figure1showst hatt hebluestarsareclosert ot hezero ilnet han .
e l c r i c d e r e h
t Theextremevalueofthepredicitonerrorsofthe ilnearrobus tSVRare e h t f o e u l a v e m e r t x e e h t ; 8 5 2 0 . 0 s i e u l a v e t u l o s b a e g a r e v a s ti d n a , r
pediciton errors of the ilnear robus tSVR are ,and tis average e
h t , e r o m r e h t r u F . 9 6 4 1 . 0 s i e u l a v e t u l o s b
a predicitonerroroft he ilnearrobus tSVRa t
d n a t n i o p g n it s a c e r o f e h t n e e w t e b e c n a t s i d e h t o t l a n o it r o p o r p y l e s r e v n i s i t n i o p e m o s
t e h
t raining poin ta tX axis .Therefore ,the prediciton accuracy of the ilnear robus t r a e n il e h t o t d e r a p m o c e d u ti n g a m f o r e d r o e n o y b d e v o r p m i s i m h ti r o g l a R V S
. n o it a b r u t r e p l a r d e h y l o p h ti w a t a d t u p n i g n i s s e c o r p f o t c e p s e r n i R V S d r a d n a t s
In figure 2 ,the blue stars are the prediciton errors of the nonilnear robus tSVR .
m h ti r o g l
a The red circles are the prediciton errors of the nonilnear standard SVR .
m h ti r o g l
a Comparingt ot hefigure1,t hebluestarsareclosert ot hezero ilnet hant he .
s e l c r i c d e
r The extreme value of the prediciton errors of the ilnear robus tSVR are e h t f o e u l a v e m e r t x e e h t ; 5 5 5 3 . 0 s i e u l a v e t u l o s b a e g a r e v a s ti d n a , n
o it c i d e r
p errors ofthe ilnear robus tSVR are ,and tis average
e r p e h T . 5 8 6 4 . 3 s i e u l a v e t u l o s b
a diciton accuracy of the nonilnear robus tSVR
r a e n il n o n e h t o t d e r a p m o c e d u ti n g a m f o r e d r o o w t r o e n o y b d e v o r p m i s i m h ti r o g l a
. R V S d r a d n a t s
N O I S U L C N O C
Int hispaper ,arobus tsuppor tvectorregressionmethodwasi ntroduced,i ncluding l
e d o m l a c it a m e h t a m e h
t ,the common kerne lfuncitons and the mos twidely used n
a i s s u a
G funciton . Moreover , ilnear regression and nonilnear regression algortihm e
r e w w o l
f analyz de . To vaildate the effecitveness of the robus tsuppor t vector n
o i s s e r g e
r method ,two numerica lexperiments were conducted .The resutls indicate o w t r o e n o y b d e v o r p m i s i m h ti r o g l a R V S t s u b o r e h t f o y c a r u c c a n o it c i d e r p e h t t a h t
d n a n o i s s e r g e r r a e n il h t o b n i R V S d r a d n a t s e h t o t d e r a p m o c e d u ti n g a m f o r e d r o
. s e l p m a x e n o i s s e r g e r r a e n il n o n
S T N E M E G D E L W O N K C A
The Projec tSupported by Natura lScience Basic Research Plan in Shaanx i (
a n i h C f o e c n i v o r
P GrantNo.2016JQ5054) ,ScienceResearch andDevelopmen tPlan (
a n i h C f o e c n i v o r P i x n a a h S n
i Grant No.2016GY-007) ,Research Fund for the
i X f o m a r g o r P l a r o t c o
D ’anUniverstiyofScienceandTechnologyi nShaanx iProvince (
a n i h C f
o GrantNo.2015QDJ053)andFosterFundofXi’anUniverstiyofScienceand (
a n i h C f o e c n i v o r P i x n a a h S n i y g o l o n h c e
T GrantNo.201629) .Inaddiiton,t heauthors
f s a e d i t a e r g y b d e r i p s n i e r a r e p a p s i h t f
o romZhixiaYang ,YingijeTianandNaiyang
, g n e
D theirworksprovideagrea tassistanceforus.Weareheret oexpresso ursincere e
d u ti t a r
S E C N E R E F E R
.
1 TomMtichell .MachineLearning[M] .McGrawHill ,1997.
.
2 VladimirVapnik .TheNatureofStaitsitca lLearningTheory[M] .NewYork :Springer ,1999.
.
3 Soyster A.L .Convex Programming wtih Set-inclusive Constraints and Appilcaitons to Inexac t
4 5 1 1 : ) 5 ( 1 2 , 3 7 9 1 , h c r a e s e R s n o it a r e p O . ] J [ g n i m m a r g o r P r a e n i
L -1157.
.
4 CorinnaCortes ,VladimirVapnik .Support-VectorNetwork s[J] .MachineLearning ,1995 ,20(3) :
3 7
2 -297.
.
5 John Platt .Fas ttraining ofsuppor tvectormachines using sequenita lminima lopitmizaiton [M] .
. 9 9 9 1 , t r o p e R l a c i n h c e T h c r a e s e R t f o s o r c i M .
6 NelloCrisitanini, JohnShawetaylor .Anintroducitontosuppor tvectormachinesandotherkernel
-s d o h t e m g n i n r a e l d e s a
b [M] .CambridgeUniverstiyPress ,2000.
.
7 Jie Liu ,Enrico Zio .An Adapitve Onilne Learning Approach for Suppor tVector Regression :
e n il n
O -SVR-FID[J] .Mechanica lSystemsandSigna lProcessing ,2016.
.
8 ZhixiaYang ,YingjieTian ,Naiyang Deng .Second OrderConeProgramming Formulaitons for
n o m u i s o p m y S l a n o it a n r e t n I t s r i F e h T . ] J [ e n i h c a M n o i s s e r g e R l a n i d r O r o t c e V t r o p p u S t s u b o R
2 3 3 : 3 1 0 2 , y g o l o i B s m e t s y S d n a n o it a z i m it p
O -340.
.
9 Yuan Lv ,Zhong Gan .Robus tε - Support Vector Regression [J] .Mathemaitca lProblems in
