1 Hilbert Space and Kernel An inner product uv can be 1 a usual dot product uv 2 a kernel product uv vw where may have in64257nite dimensions However an inner product must satisfy the following conditions 1 Symmetry uv vu uv 8712 X 2 Bilinearity ID: 24763
Download Pdf The PPT/PDF document "CSBStatB Spring Statistical Learning Th..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
2ReproducingKernelHilbertSpacesAReproducingKernelHilbertSpace(RKHS)isaHilbertspaceHwithareproducingkernelwhosespanisdenseinH.WecouldequivalentlydeneanRKHSasaHilbertspaceoffunctionswithallevaluationfunctionalsboundedandlinear.Forinstance,theL2spaceisaHilbertspace,butnotanRKHSbecausethedeltafunctionwhichhasthereproducingpropertyf(x)=Zs(x�u)f(u)dudoesnotsatisfythesquareintegrablecondition,thatis,Zs(u)2du61;thusthedeltafunctionisnotinL2.Nowletusdeneakernel.Denition.k:XX!Risakernelif1.kissymmetric:k(x;y)=k(y;x).2.kispositivesemi-denite,i.e.,8x1;x2;:::;xn2X,the"GramMatrix"KdenedbyKij=k(xi;xj)ispositivesemi-denite.(AmatrixM2Rnnispositivesemi-deniteif8a2Rn,a0Ma0.)Herearesomepropertiesofakernelthatareworthnoting:1.k(x;x)0.(ThinkabouttheGrammatrixofn=1)2.k(u;v)p k(u;u)k(v;v).(ThisistheCauchy-Schwarzinequality.)Toseewhythesecondpropertyholds,weconsiderthecasewhenn=2:Leta=k(v;v)�k(u;v).TheGrammatrixK=k(u;u)k(u;v)k(v;u)k(v;v)0()a0Ka0()[k(v;v)k(u;u)�k(u;v)2]k(v;v)0.Bytherstpropertyweknowk(v;v)0,sok(v;v)k(u;u)k(u;v)2.1.2BuildanReproducingKernelHilbertSpace(RKHS)Givenakernelk,denethe"reproducingkernelfeaturemap":X!RXas:(x)=k(;x)Considerthevectorspace:span(f(x):x2Xg)=ff()=nXi=1 ik(;xi):n2N;xi2X; i2RgForf=Pi ik(;ui)andg=Piik(;vi),denehf;gi=Pi;j ijk(ui;vj).Notethat:hf;k(;x)i=Xi ik(x;ui)=f(x),i.e.,khasthereproducingproperty.Weshowthathf;giisaninnerproductbycheckingthefollowingconditions: 4ReproducingKernelHilbertSpacesTotakeananalogueinthenitecase,thatis,X=fx1;:::;xng.LetKij=k(xi;xj),andf:X!Rnwithfi=f(xi).Then,Tkf=nXi=1k(;xi)fi8f;f0Kf0)K0)K=Xiviv0iHence,k(xi;xj)=Kij=(VV0)ij=nXk=1kvkivkj=nXk=1k k(xi) k(xj)) k(xi)=(vk)iWesummarizeseveralequivalentconditionsoncontinuous,symmetrickdenedoncompactX:1.EveryGrammatrixispositivesemi-denite.2.Tkispositivesemi-denite.3.kcanbeexpressedask(u;v)=Pii i(u) i(v).4.kisthereproducingkernelofanRKHSoffunctionsonX.