Y0xPY1Xwexpw0iwiXiPY0PY1thtrainingexampleandwhereYykis1ifYykand0otherwiseNotetheroleofhereistoselectonlythosetrainingexamplesforwhichYykThemaximumlikelihoodestimatorfor2ikis2ik1jYjykjXjiik2Yjyk14Thism ID: 871429
Download Pdf The PPT/PDF document "3F240A12JENSLG104Vmpgcylindersdisplaceme..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1 3F?'2)4'=0A12'JENSLG'@+10.4=?V'mpgcylind
3F?'2)4'=0A12'JENSLG'@+10.4=?V'mpgcylindersdisplacementhorsepowerweightaccelerationmodelyearmakergood49775226518.277asiaame
2 ricabad4121110260012.877europebad8350175
ricabad4121110260012.877europebad835017541001373americabad619895310216.574americabad410894237916.573asia (Y=0 !x)P(Y=1|X,w)
3 !exp(w0+!iwiXi)P(Y=0 P(Y=1 thtrainingexa
!exp(w0+!iwiXi)P(Y=0 P(Y=1 thtrainingexample,andwhere!(Y=yk)is1ifY=ykand0otherwise.Notetheroleof!hereistoselectonlythosetra
4 iningexamplesforwhichY=yk.Themaximumlike
iningexamplesforwhichY=yk.Themaximumlikelihoodestimatorfor"2ikisö"2ik=1#j!(Yj=yk)#j(Xji"öµik)2!(Yj=yk)(14)Thismaximumlikeli
5 hoodestimatorisbiased,sotheminimumvarian
hoodestimatorisbiased,sotheminimumvarianceunbi-asedestimator(MVUE)issometimesusedinstead.Itisö"2ik=1(#j!(Yj=yk))"1#j(Xji"öµ
6 ik)2!(Yj=yk)(15)3LogisticRegressionLogis
ik)2!(Yj=yk)(15)3LogisticRegressionLogisticRegressionisanapproachtolearningfunctionsoftheformf:X#Y,orP(Y|X)inthecasewhereYi
7 sdiscrete-valued,andX=$X1...Xn%isanyvect
sdiscrete-valued,andX=$X1...Xn%isanyvectorcontainingdiscreteorcontinuousvariables.Inthissectionwewillprimarilycon-siderthec
8 asewhereYisabooleanvariable,inordertosim
asewhereYisabooleanvariable,inordertosimplifynotation.IntheÞnalsubsectionweextendourtreatmenttothecasewhereYtakesonanyÞnite
9 numberofdiscretevalues.LogisticRegressio
numberofdiscretevalues.LogisticRegressionassumesaparametricformforthedistributionP(Y|X),thendirectlyestimatesitsparametersf
10 romthetrainingdata.Theparametricmodelass
romthetrainingdata.TheparametricmodelassumedbyLogisticRegressioninthecasewhereYisbooleanis:P(Y=1|X)=11+expexp(w0+#ni=1wiXi)
11 1+expNoticethatequation(17)followsdirect
1+expNoticethatequation(17)followsdirectlyfromequation(16),becausethesumofthesetwoprobabilitiesmustequal1.Onehighlyconvenie
12 ntpropertyofthisformforP(Y|X)isthatitlea
ntpropertyofthisformforP(Y|X)isthatitleadstoasimplelinearexpressionforclassiÞcation.ToclassifyanygivenXwegenerallywanttoass
13 ignthevalueykthatmaximizesP(Y=yk|X).Puta
ignthevalueykthatmaximizesP(Y=yk|X).Putanotherway,weassignthelabelY=0ifthefollowingconditionholds:1P(Y=0|X)P(Y=1|X)substitu
14 tingfromequations(16)and(17),thisbecomes
tingfromequations(16)and(17),thisbecomes1expCopyrightc!2010,TomM.Mitchell.8!50500.20.40.60.81YXY = 1/(1 + exp(!X))Figure1:F
15 ormofthelogisticfunction.InLogisticRegre
ormofthelogisticfunction.InLogisticRegression,P(Y|X)isas-sumedtofollowthisform.andtakingthenaturallogofbothsideswehavealine
16 arclassiÞcationrulethatassignslabelY=0if
arclassiÞcationrulethatassignslabelY=0if n#,whereeachXiisacontinuousrandomvariablew.X+w0 = 0 1P(Y=1|X)P(Y=0|X)P(Y=0 (Y=0 X)
17 "exp(w10+!iw1iXi)P(Y=2|X)"exp(w20+!iw2iX
"exp(w10+!iw1iXi)P(Y=2|X)"exp(w20+!iw2iXi)P(Y=r|X)=1!r!1!j=1 X)"exp(w10+!iw1iXi)P(Y=2|X)"exp(w20+!iw2iXi)P(Y=r|X)=1!r!1!j=1
18 w0+PiwiXi!l(w)!w=!jxji$yj!P(Yj=1|xj,w)%
w0+PiwiXi!l(w)!w=!jxji$yj!P(Yj=1|xj,w)%!l(w)!w=!j&!!wyj(w0+!iwixji)!!!wln"1+exp(w0+!iwixji)#'=!jyjxji3error=!i(ti!öti)2=!i
19 "ti!!kwkhk(xi)#2P(Y=1 X)"exp(w10+!iw1iXi
"ti!!kwkhk(xi)#2P(Y=1 X)"exp(w10+!iw1iXi)P(Y=2|X)"exp(w20+!iw2iXi)P(Y=r|X)=1!r!1!j=1P(Y=j|X)=!jyjlnew0+PiwiXi1+ew0+PiwiXi+(
20 1!yj)ln11+ew0+PiwiXi!l(w)!wi=!jxji$yj!P(
1!yj)ln11+ew0+PiwiXi!l(w)!wi=!jxji$yj!P(Yj=1|xj,w)%!l(w)!wi=!j&!!wyj(w0+!iwixji)!!!wln"1+exp(w0+!iwixji)#'=!j&yjxji!xjiexp(
21 w0+(iwixji)1+exp(w0+(iwixji)'=!jxji&yj!e
w0+(iwixji)1+exp(w0+(iwixji)'=!jxji&yj!exp(w0+(iwixji)1+exp(w0+(iwixji)'11+ !iw2i!lnp(w)!wi=!"wi1#i#2$e!(x!µik MTGG'd'7E7c6
22 \8G'd'6\8'+d6M'[d7&'U67E?7cJEN7d7SU7MTGG
\8G'd'6\8'+d6M'[d7&'U67E?7cJEN7d7SU7MTGG'd'7E6c6\8G'd'c6\8' MTGG'd'c5E6c6\8G'd'7\6'd5M'[d6&'U56E?6cJEN6d7SU6MTGG'd'cbE7c6\8
23 G'd'c7\8'+d5M'[d7&'U57E?7cJEN7d7SU7MTGG'
G'd'c7\8'+d5M'[d7&'U57E?7cJEN7d7SU7MTGG'd'5E6c6\8G'd'c7\6' /0*9=A1+OA-)2&'B02A=+O0'F+*F'T0+*F4,'Ð!A*A+2M'�)10')2'4F+
24 ,'=A401'+2'4F0'p9A1401'plot1!"1!e^"x#fro
,'=A401'+2'4F0'p9A1401'plot1!"1!e^"x#from"5to5thickInputinterpretation:Show$plot1#"x!1x!"5to5%Result:"4"2240.20.40.60.81.0
25 Generated by Wolfram|Alpha (www.wolframa
Generated by Wolfram|Alpha (www.wolframalpha.com) on January 30, 2012 from Champaign, IL.© Wolfram Alpha LLCÑA Wolfram Rese
26 arch Company1plot1!"1!e^""2x##from"5to5t
arch Company1plot1!"1!e^""2x##from"5to5thickInputinterpretation:Show$plot1#"2x!1x!"5to5%Result:"4"2240.20.40.60.81.0 Genera
27 ted by Wolfram|Alpha (www.wolframalpha.c
ted by Wolfram|Alpha (www.wolframalpha.com) on January 30, 2012 from Champaign, IL.© Wolfram Alpha LLCÑA Wolfram Research C
28 ompany1plot1!"1!e^""10x##from"5to5thickI
ompany1plot1!"1!e^""10x##from"5to5thickInput !1x!"5to5%Result:"4"2240.20.40.60.81.0 Generated by Wolfram|Alpha (www.wolfram
29 alpha.com) on January 30, 2012 from Cham
alpha.com) on January 30, 2012 from Champaign, IL.© Wolfram Alpha LLCÑA Wolfram Research Company1error=!i(ti!öti)2=!i"ti!!k
30 wkhk(xi)#2P(Y=1 !wi=!"wi3error=!i(ti!öti
wkhk(xi)#2P(Y=1 !wi=!"wi3error=!i(ti!öti)2=!i"ti!!kwkhk(xi)#2P(Y=1 d6\7' ()*+,-.'/0*10,,+)2'BA1�A0401'0,-�A40