/
Chapter 4 Basic Probability and Probability Distributions Chapter 4 Basic Probability and Probability Distributions

Chapter 4 Basic Probability and Probability Distributions - PowerPoint Presentation

natalia-silvester
natalia-silvester . @natalia-silvester
Follow
442 views
Uploaded On 2018-03-20

Chapter 4 Basic Probability and Probability Distributions - PPT Presentation

Probability Terminology Classical Interpretation Notion of probability based on equal likelihood of individual possibilities coin toss has 12 chance of Heads card draw has 452 chance of an Ace Origins in games of chance ID: 657806

probability distribution event probabilities distribution probability probabilities event random values table distributions sample normal step standard variable obtain interest

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Chapter 4 Basic Probability and Probabil..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Chapter 4

Basic Probability and Probability DistributionsSlide2

Probability Terminology

Classical Interpretation

: Notion of probability based on equal likelihood of individual possibilities (coin toss has 1/2 chance of Heads, card draw has 4/52 chance of an Ace). Origins in games of chance.

Outcome

: Distinct result

of random process (

N

= # outcomes)

Event

: Collection of outcomes (

N

e

= # of outcomes in event)

Probability of event

E

:

P

(event

E

) =

N

e

/

N

Relative Frequency Interpretation

: If an experiment were conducted repeatedly, what fraction of time would event of interest occur (based on empirical observation)

Subjective Interpretation

: Personal view (possibly based on external info) of how likely a one-shot experiment will end in event of interest Slide3

Obtaining Event Probabilities

Classical Approach

List all

N

possible outcomes of experiment

List all

N

e

outcomes corresponding to event of interest (

E

)

P

(event

E

) =

N

e

/

N

Relative Frequency Approach

Define event of interest

Conduct experiment repeatedly (often using computer)

Measure the fraction of time event

E

occurs

Subjective Approach

Obtain as much information on process as possible

Consider different outcomes and their likelihood

When possible, monitor your skill (e.g. stocks, weather)Slide4

Basic Counting RulesSlide5

Basic Probability and Rules

A,B

 Events of interest

P

(

A

),

P

(

B

)  Event probabilities

Union

: Event

either

A

or

B

occurs

(

A

B

)

Mutually Exclusive

:

A, B

cannot occur at same time

If

A,B

are mutually exclusive:

P

(either

A

or

B

) =

P

(

A

) +

P

(

B

)

Complement of

A

: Event that

A

does not occur (

Ā

)

P

(

Ā

) = 1-

P

(

A

) That is:

P

(

A

) +

P

(

Ā

) = 1

Intersection

: Event

both

A

and

B

occur

(

A

B

or

AB

)

P

(

A

B

) =

P

(

A

) +

P

(

B

) -

P

(

AB

) Slide6

Conditional Probability and Independence

Unconditional/Marginal Probability

: Frequency which event occurs in general (given no additional info).

P

(

A

)

Conditional Probability

: Probability an event (

A

) occurs

given

knowledge another event (B) has occurred. P(A|B)Independent Events: Events whose unconditional and conditional (given the other) probabilities are the sameSlide7

John Snow London Cholera Death Study

2 Water Companies (Let

D

be the event of death):

Southwark&Vauxhall (

S

): 264913 customers, 3702 deaths

Lambeth (

L

): 171363 customers, 407 deaths

Overall: 436276 customers, 4109 deaths

Note that probability of death is almost 6 times higher for S&V customers than Lambeth customers (was important in showing how cholera spread)Slide8

John Snow London Cholera Death Study

Contingency Table with joint probabilities (in body of table) and marginal probabilities (on edge of table)Slide9

John Snow London Cholera Death Study

WaterUser

S&V

L

.6072

.3928

Company

Death

D (.0085)

.0140

.9860

D

C

(.5987)

.0024

.9976

D (.0009)

D

C

(.3919)

Tree Diagram obtaining joint probabilities by multiplication ruleSlide10

Bayes’s Rule - Updating Probabilities

Let

A

1

,…,

A

k

be a set of events that

partition

a sample space such that (mutually exclusive and exhaustive):

each set has known

P(A

i) > 0 (each event can occur)for any 2 sets Ai and Aj, P(Ai

and Aj) = 0 (events are disjoint)P(A1) + … +

P(A

k

) =

1 (each outcome belongs to one of events)

If

C

is an event such that

0 <

P(C)

< 1 (

C

can occur, but will not necessarily occur)

We know the probability will occur given each event

Ai

:

P(C|A

i

)Then we can compute probability of Ai given C occurred:Slide11

Northern Army at Gettysburg

Regiments: partition of soldiers (

A

1

,…,

A

9

). Casualty: event

C

P(A

i

) = (size of regiment) / (total soldiers) = (Column 3)/95369

P(C|A

i

) =

(# casualties) / (regiment size) = (Col 4)/(Col 3)

P(C|A

i

) P(A

i

) =

P(A

i

and

C)

= (Col 5)*(Col 6)

P(C)

=sum(Col 7)

P(A

i|C) =

P(Ai and C) / P(C) = (Col 7)/.2416Slide12

Example - OJ Simpson Trial

Given Information on Blood Test (

T

+/

T

-)

Sensitivity:

P

(

T

+|Guilty)=1

Specificity:

P(T-|Innocent)=.9957  P(T+|I)=.0043

Suppose you have a prior belief of guilt: P(G)=p*What is “posterior” probability of guilt after seeing evidence that blood matches:

P

(

G

|

T

+)?

Source: B.Forst (1996). “Evidence, Probabilities and Legal Standards for Determination of Guilt: Beyond the OJ Trial”, in

Representing OJ: Murder, Criminal Justice, and the Mass Culture

, ed. G. Barak pp. 22-28. Harrow and Heston, Guilderland, NYSlide13
Slide14

Random Variables/Probability Distributions

Random Variable

: Outcome characteristic that is not known prior to experiment/observation

Qualitative Variables

: Characteristics that are non-numeric (e.g. gender, race, religion, severity)

Quantitative Variables

: Characteristics that are numeric (e.g. height, weight, distance)

Discrete

: Takes on only a countable set of possible values

Continuous

: Takes on values along a continuum

Probability Distribution

: Numeric description of outcomes of a random variable takes on, and their corresponding probabilities (discrete) or densities (continuous)Slide15

Discrete Random Variables

Discrete RV: Can take on a finite (or countably infinite) set of possible outcomes

Probability Distribution: List of values a random variable can take on and their corresponding probabilities

Individual probabilities must lie between 0 and 1

Probabilities sum to 1

Notation:

Random variable:

Y

Values

Y

can take on:

y

1, y2, …, ykProbabilities: P(

Y=y1) = p1 …

P

(

Y

=

y

k

) =

p

k

p

1

+ … +

pk = 1Slide16

Example: Wars Begun by Year (1482-1939)

Distribution of Numbers of wars started by year

Y

= # of wars stared in randomly selected year

Levels:

y

1

=0,

y

2

=1,

y

3=2, y4=3, y5=4Probability Distribution:Slide17

Masters Golf Tournament 1st Round ScoresSlide18

Means and Variances of Random Variables

Mean: Long-run average

a random variable will take on (also the balance point of the probability distribution)

Expected Value is another term, however we really do not expect that a realization of

X

will necessarily be close to its mean. Notation:

E

(

X

)

Mean and Variance of a discrete random variable:Slide19

Rules for Means

Linear Transformations:

a

+

bY

(where

a

and

b

are constants):

E

(

a+bY) = ma+bY = a + bmYSums of random variables: X + Y (where X and

Y are random variables): E(X+Y) = m

X+Y

=

m

X

+

m

Y

Linear Functions of Random Variables:

E

(

a

1

Y

1

+

+

anYn) =

a1m1

+…+anmn where E(Yi)=miSlide20

Example: Masters Golf Tournament

Mean by Round (Note ordering):

m

1

=73.54

m

2

=

73.07

m

3

=73.76

m4=73.91Mean Score per hole (18) for round 1: E((1/18)X

1) = (1/18)m1 = (1/18)73.54 = 4.09Mean Score versus par (72) for round 1:

E

(

X

1

-72) =

m

X1-72

= 73.54-72= +1.54 (1.54 over par)

Mean Difference (Round 1 - Round 4):

E

(

X

1

-

X

4) = m1 - m4 = 73.54 - 73.91 = -0.37

Mean Total Score: E(X1+X

2+X3+X4) = m1+ m2+ m3+ m4 = = 73.54+73.07+73.76+73.91 = 294.28 (6.28 over par)Slide21

Variance of a Random Variable

Special Cases:

X

and

Y

are independent (outcome of one does not alter the distribution of the other):

r

= 0, last term drops out

a=b=

1 and

r

= 0

V

(

X+Y

) =

s

X

2

+

s

Y

2

a=1 b= -

1 and

r

= 0

V

(

X-Y

) =

s

X2 +

sY2 a=b=1 and r 0 V(X+Y) = sX2 + sY2 + 2

rsXsY a=1 b= -1 and r 0 V(X-Y) = sX2 + sY

2 -2rsX

sYSlide22

Examples - Wars & Masters Golf

m

=0.67

m

=73.54Slide23

Binomial Distribution for Sample Counts

Binomial “Experiment”

Consists of

n

trials or observations

Trials/observations are independent of one another

Each trial/observation can end in one of two possible outcomes often labelled “Success” and “Failure”

The probability of success,

p

, is constant across trials/observations

Random variable,

Y

, is the number of successes observed in the n trials/observations. Binomial Distributions: Family of distributions for Y, indexed by Success probability (p) and number of trials/observations (

n). Notation: Y~B(n,p)Slide24

Binomial Distributions and Sampling

Problem when sampling from a finite population: the sequence of probabilities of Success is altered after observing earlier individuals.

When the population is much larger than the sample (say at least 20 times as large), the effect is minimal and we say

X

is approximately binomial

Obtaining probabilities:Slide25

Example - Diagnostic Test

Test claims to have a sensitivity of 90% (Among people with condition, probability of testing positive is .90)

10 people who are known to have condition are identified,

Y

is the number that correctly test positive

Table obtained in EXCEL with function:

BINOM.DIST(

k,n,

p

,

FALSE

)

(TRUE option gives cumulative distribution function:

P

(

Y

k

)Slide26

Binomial Mean & Standard Deviation

Let

S

i

=1 if the i

th

individual was a success, 0 otherwise

Then

P

(

S

i

=1) = p and P(Si=0) = 1-pThen E

(Si)=mS = 1(

p

) + 0(1-

p

) =

p

Note that

Y = S

1

+…+

S

n

and that trials are independent

Then

E

(

Y

)=

mY = nmS = n

pV(Si) = E

(Si2)-mS2 = p-p2 = p(1-p)Then V(Y)=sY2 = np

(1-p)Slide27

Poisson Distribution for Event Counts

Distribution related to Binomial for Counts of number of events occurring in fixed time or space. Takes many “sub-intervals” and assumes Binomial (

n

=1) distribution for events in each.

The average number of events in unit time or space is

m

. In general, for length

t

, mean is

m

t

Table 14 (pp. 1121-1123) gives probabilities for selected

m

EXCEL Function: =POISSON.DIST(

y

,

m

,0) returns

P

(

y

)Slide28

Poisson Distribution for Event CountsSlide29

Continuous Random Variables

Variable can take on any value along a continuous range of numbers (interval)

Probability distribution is described by a smooth density curve

Probabilities of ranges of values for

Y

correspond to areas under the density curve

Curve must lie on or above the horizontal axis

Total area under the curve is 1

Special cases: Normal and Gamma distributionsSlide30

Normal Distribution

Bell-shaped, symmetric family of distributions

Classified by 2 parameters: Mean (

m

) and standard deviation (

s

). These represent

location

and

spread

Random variables that are approximately normal have the following properties wrt individual measurements:

Approximately half (50%) fall above (and below) mean

Approximately 68% fall within 1 standard deviation of meanApproximately 95% fall within 2 standard deviations of meanVirtually all fall within 3 standard deviations of meanNotation when Y is normally distributed with mean m

and standard deviation s :Slide31

Two Normal DistributionsSlide32

Normal DistributionSlide33

Example - Heights of U.S. Adults

Female and Male adult heights are well approximated by normal distributions:

Y

F

~N(63.7,2.5)

Y

M

~N(69.1,2.6)

Source:

Statistical Abstract of the U.S.

(1992)Slide34

Standard Normal (

Z

) Distribution

Problem: Unlimited number of possible normal distributions (-

 <

m

<  ,

s

> 0)

Solution: Standardize the random variable to have mean 0 and standard deviation 1

Probabilities of certain ranges of values and specific percentiles of interest can be obtained through the standard normal (

Z

) distributionSlide35
Slide36

Standard Normal (Z) Distribution

z

Table Area

1-Table AreaSlide37

I

n

t

g

e

r

p

a

r

t

&

1st

D

e

c

i

m

a

l

2nd Decimal PlaceSlide38

2nd Decimal Place

I

n

t

g

e

r

p

a

r

t

&

1st

D

e

c

i

m

a

lSlide39

Finding Probabilities of Specific Ranges

Step 1 -

Identify the normal distribution of interest (e.g. its mean (

m

) and standard deviation (

s

) )

Step 2 -

Identify the range of values that you wish to determine the probability of observing (

y

L

,

yU), where often the upper or lower bounds are  or -Step 3 - Transform y

L and yU into

Z

-values:

Step 4 -

Obtain P(

z

L

Z

z

U

) from

Z

-table (pp. 1086-7)Slide40

Example - Adult Female Heights

What is the probability a randomly selected female is 5’10” or taller (70 inches)?

Step 1 -

Y

~ N(63.7 , 2.5)

Step 2 -

y

L

= 70.0

y

U = Step 3 -

Step 4 -

P(

Y

 70) = P(

Z

 2.52) =

1-P(Z2.52)=1-.9941=.0059 (  1/170) Slide41

Finding Percentiles of a Distribution

Step 1 -

Identify the normal distribution of interest

(e.g. its mean (

m

) and standard deviation (

s

) )

Step 2

-

Determine the percentile of interest 100

p% (e.g. the 90th percentile is the cut-off where only 90% of scores are below and 10% are above).Step 3 - Find p in the body of the z-table and itscorresponding z-value

(zp) on the outer edge:If 100p < 50 then use left-hand page of table

If 100

p

50 then use right-hand page of table

Step 4

- Transform

z

p

back to original units:Slide42

Example - Adult Male Heights

Above what height do the tallest 5% of males lie above?

Step 1 -

Y

~ N(69.1 , 2.6)

Step 2 -

Want to determine 95

th

percentile (

p

= .95)

Step 3 -

P(Z1.645) = .95Step 4 - y.95 = 69.1 + (1.645)(2.6) = 73.4 (6’,1.4”)Slide43

Assessing Normality and Transformations

Obtain a histogram and see if mound-shaped

Obtain a normal probability plot

Order data from smallest to largest and rank them (1 to

n

)

Obtain a percentile for each: pct = (rank-0.375)/(

n

+0.25)

Obtain the

z

-score corresponding to the percentile

Plot observed data versus z-score, see if straight line (approx.)Transformations that can achieve approximate normality:Slide44

Chi-Square Distribution

Indexed by “degrees of freedom (

n

)” X~

c

n

2

Z~N(0,1)

 Z

2

~

c

12Assuming Independence:

Obtaining Probabilities in EXCEL:

To obtain: 1-F(x)=P(

X≥x

) Use Function: =CHISQ.DIST.RT(

x,

n

)

Table 7 (pp. 1095-6) Gives critical values for selected upper tail probabilitiesSlide45

Chi-Square DistributionsSlide46

Critical Values for Chi-Square Distributions (Mean=

n

, Variance=2

n

)Slide47

Student’s t-Distribution

Indexed by “degrees of freedom (

n

)” X~t

n

Z~N(0,1), X~

c

n

2

Assuming Independence of Z and X:

Obtaining Probabilities in EXCEL:

To obtain: 1-F(t)=P(

T≥t

) Use Function: =T.DIST.RT(

t,

n

)

Table 2 (p. 1088 gives critical values for selected upper tail

probsSlide48
Slide49

Critical Values for Student’s t-DistributionsSlide50

F-Distribution

Indexed by 2 “degrees of freedom (

n

1

,n

2

)” W~F

n1,n2

X

1

~

c

n12, X2

~cn22Assuming Independence of X

1

and X

2

:

Obtaining Probabilities in EXCEL:

To obtain: 1-F(w)=P(

W≥w

) Use Function: =F.DIST.RT(w,

n

1

,n

2

)

Table 8 (pp. 1097-1108) gives critical values for selected upper tail

probsSlide51
Slide52

Critical Values for F-distributions P(F ≤ Table Value) = 0.95Slide53

Sampling Distributions

Distribution of a Sample Statistic: The probability distribution of a sample statistic obtained from a random sample or a randomized experiment

What values can a sample mean (or proportion) take on and how likely are ranges of values?

Population Distribution: Set of values for a variable for a population of individuals. Conceptually equivalent to probability distribution in sense of selecting an individual at random and observing their value of the variable of interestSlide54

Sampling Distribution of a Sample Mean

Obtain a sample of

n

independent measurements of a quantitative variable:

Y

1

,…,

Y

n

from a population with mean

m

and standard deviation

sAverages will be less variable than the individual measurementsSampling distributions of averages will become more like a normal distribution as n increases (regardless of the shape of the population of individual measurements)Slide55

Central Limit Theorem

When random samples of size

n

are selected from any population with mean

m

and finite standard deviation

s

, the sampling distribution of the sample mean will be approximately distributed for large

n

:

Z-

table can be used to approximate probabilities of ranges of values for sample means, as well as percentiles of their sampling distribution Slide56

Sample Proportions

Counts of Successes (

Y

) rarely reported due to dependency on sample size (

n

)

More common is to report the

sample proportion

of successes:Slide57

Sampling Distributions for Counts & Proportions

For samples of size

n

, counts (and thus proportions) can take on only

n

distinct possible outcomes

As the sample size

n

gets large, so do the number of possible values, and sampling distribution begins to approximate a normal distribution. Common Rule of thumb:

n

p

 10 and n(1-p)  10 to use normal approximationSlide58

Sampling Distribution for

Y

~

B

(

n

=1000,

p=0.2)Slide59

Using

Z

-Table for Approximate Probabilities

To find probabilities of certain ranges of counts or proportions, can make use of fact that the sample counts and proportions are approximately normally distributed for large sample sizes.

Define range of interest

Obtain mean of the sampling distribution

Obtain standard deviation of sampling distribution

Transform range of interest to range of

Z

-values

Obtain (approximate) Probabilities from

Z-

table