100 samples ~ Beta(5,2) - PDF document

100 samples ~ Beta(5,2) Normal fit Beta fit the graphs, the distribution \ftting to ol outputs the follo wing information: Distribution: Normal Log likelihood: 55.2571 7 100 samples ~ Beta(5,2) Normal fit Beta fit 0.8 2.5 0.9 3 0.7 0.6 Cumulative probability 2 Density 0.5 0.4 0.3 1.5 1 XXXXXX Y and log-likelihood function is n 1(Xi � ) 2 2 2 log '(; 2 ) = log p 2 � log  � i=1 n 11 = n log p 2 � n log  � (Xi � ) 2 : 2 2 We want to maximize the log-likelihood with respect to �1  1 and  2 � 0: First, obviously, for any  we need to minimize P (Xi � ) 2 over : The critical point condition is nn X i =1 d (Xi � ) 2 = � 2 (Xi � )=0 d i=1 i=1  and solving this for  we get that ^ = X: We can plug this estimate in the log-likelihood and it remains to maximize n 11 X  ) 2 n log p 2 � n log  � (Xi � 2 2 i=1 over : The critical point condition reads, n 1 (Xi � X  ) 2 =0 � +  3 and solving this for  we obtain that the MLE of  2 is n 1 ^ 2 X  ) 2 = (X i � n : i=1 The normal distribution \ft in \fgure 2.1 corresponds to these parameters (^;^ 2 ): Exercise. Generate a normal sample in Matlab and \ft it with a normal distribution using 'd\fttool'. Then plot a p.d.f. or c.d.f. corresponding to MLE above and compare this with 'd\fttool'. Let us give one more example of MLE. Uniform distribution U[0;] on the interval [0;]. This distribution has p.d.f. 1 ; 0  x  ; otherwise. f(x)= j 0; The likelihood function n 1 '() = i=1 f(Xij) = = n I(X1; : : : ; Xn 2 [0; ]) 1 n I(max(X1; : : : ; Xn)  ): 13 PSfragreplacements Here the indicator function I(A) equals to 1 if event A happens and 0 otherwise. What the indicator above means is that the likelihood will be equal to 0 if at least one of the factors is 0 and this will happen if at least one observation X i will fall outside of the 'allowed' interval [0;]: Another way to say it is that the maximum among observations will exceed ; i.e. '()=0 if  max(X1;:::;Xn); and 1 '()= if   max(X1;:::;Xn): n Therefore, looking at the \fgure 2.3 we see that  ^ = max(X 1;:::;Xn) is the MLE. '() 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 2 2.5 3 max(X 1;:::;Xn)  Figure 2.3: MLE for the uniform distribution. Sometimes it is not so easy to \fnd the maximum of the likelihood function as in the examples above and one might have to do it numerically. Also, MLE does not always exist. Here is an example: let us consider uniform distribution U[0;) and de\fne the density by  1 ; 0  x ; f(x j )= 0  ; otherwise. The dierence is that we 'excluded' the point  by setting f()=0: Then the likelihood j function is n Y 1 '()= f(Xi)= I(max(X1;:::;Xn) ) n j i=1 14