Discrete Random Variables Chs. 2, 3, 4 Random Variables
Transcript Of Discrete Random Variables Chs. 2, 3, 4 Random Variables
Discrete Random Variables Chs. 2, 3, 4
• Random Variables • Probability Mass Functions • Expectation: The Mean and Variance • Special Distributions
Hypergeometric Binomial Poisson • Joint Distributions • Independence
Slide 1
Random Variables Consider a probability model (Ω, P ). Definition. A random variable is a function
X : Ω → IR. If X is a RV, let X = {X(ω) : ω ∈ Ω}, the range of X. Then X is discrete if X = {x1, x2, · · · }, (finte or infinite). Notation. If X is a random variable write
P [X ∈ B] = P ({ω : X(ω) ∈ B}) for B ⊆ IR. Definition. If X is a discrete RV, then the probability mass function of X is defined by
f (x) = P [X = x]. for x ∈ IR. Alternative Notation: fX .
Slide 2
Example
A committee of size n = 4 is selected from 5 men and 5 women. Then
Ω = combinations of 4, #Ω = 10 = 210, 4 P (A) = #A . #Ω
Let X = #women
Then
X = {0, 1, 2, 3, 4},
f (x) = 5 x
5 / 10 4−x 4
for x = 0, · · · , 4, and f (x) = 0 for other values of
x. Thus f (0) = f (4) = 5/210, f (1) = f (3) = 50/210, and f (2) = 100/210.
Slide 3
Hypergeometric Distributions
If a sample of size n is drawn w.o.r. from a box containing R ≥ 1 red tickets and N − R ≥ 1 white tickets, then the PMF of
X = #red tickets in the sample
is
f (r) = R r
N −R / N n−r n
for r = 0, · · · , n and f (x) = 0 for other x.
Def: Called Hypergeometric with parameters n, N , and R.
Slide 4
Properties of PMFs
If f is the PMF (probability mass function) of X,
then
f (x) ≥ 0 for all x,
(1)
f (x) = 0 unless x ∈ X ,
(2)
where X is the range of X, and
f (x) = 1.
(3)
x∈X
Moreover
P [X ∈ B] = f (x)
(4)
x∈B
for B ⊆ IR. Conversely, any function f that
satisfies (1), (2), and (3) is the PMF of some
random variable X.
Notes a). Henceforth ”PMF” means any function that satisfies (1), (2), and (3).
b). We can specify a model by giving X and f , subject to the first three conditions.
Slide 5
Expectation
If X is a discrete random variable with PMF f and randge X , then the expectation of X is
E(X) = xf (x),
x∈X
provided that the sum converges absolutely.
Example: Committees. In the committees example
E(X) = 0 × 5 + 1 × 50 + 2 × 100
210
210
210
+ 3 × 50 + 4 × 5
210
210
= 2.
Slide 6
Indicator Variables If A is an event, then the indicator function of A is
1 if ω ∈ A 1A(ω) = 0 otherwise.
The range is {0, 1}, f (0) = P (Ac) = 1 − P (A), f (1) = P (A),
and E(1A) = 0 × P (Ac) + 1 × P (A) = P (A).
Note: The language of RVs subsumes that of events.
Slide 7
Transformations . If X is a discrete random variable and Y = w(X), then Y has PMF
fY (y) =
fX (x).
x:w(x)=y
For example, P [X2 = y] = f (√y) + f (−√y)
for y > 0.
The Transformation Formula. If X is a discrete random variable and Y = w(X)
E(Y ) = w(x)fX(x),
(5)
x∈X
provided that the sum converges absolutely.
Slide 8
The Mean and Variance
If X has PMF f , then
E(X) = xf (x)
x∈X
depends only on f , assuming that it exists. It is also called the mean of f and/or X and denoted by µ. Thus. µ = E(X).
The variance of X is defined by
σ2 = E[(X − µ)2].
Thus
σ2 = (x − µ)2f (x).
x∈X
which depends only√of f ; and σ2 is also called the variance of f . σ = σ2 is called the standard
deviation of X and/or f.
A Useful Formula: If X is discrete with mean µ and variance σ2, then
E[X2] = σ2 + µ2
Slide 9
Binomial Distributions Independent Trials: A basic experiment with Ω0 is repeated n times. Events dependending on different trials (replications) are independent.
Examplea). Sampling with replacement. b). Gamling–e.g. coins tossing and roulette.
Basic Question. Given an event A0 ⊆ Ω0, what is the probability that A0 occurs k times in n trials? Formally, let
Ai = {A0 occurs on the ith trial}, X = 1A1 + · · · + 1An.
Answer P [X = k] = n pkqn−k k
for k = 0, · · · , n, where q = 1 − p. Def: Called Binomial with parameters n and p. The Mean and Variance: µ = np and σ2 = npq.
Slide 10
0.25
0.20
Graph Of f(k) With n = 10 and p = .5
f (k) = 10 ( 1 )10 k2
0
2
4
6
8
10
k
Slide 11
Example
Q: What is the probability that South gets no aces on at least k = 5 or n = 9 hands?
A: Let
Ai = {no aces on the ithhand}, X = 1A1 + · · · + 1A9.
Then
48
P (Ai) =
13 52
13
for i = 1, · · · , 9. So,
= .3038 = p, say
P [X = k] = 9 pk(1 − p)n−k k
for k = 0, · · · , 9, and
9
P [X ≥ 5] =
9 pk(1 − p)n−k.
k=5 k
So, P [X ≥ 5] = .1035.
Slide 12
0.15
Binomial probability
0.10
0.05
0.00
Poisson Distributions AKA The Law of Rare Events
The Derivation: As
n → ∞,
p → 0,
0 < λ = np < ∞ constant,
lim n pk(1 − p)n−k = 1 λke−λ
k
k!
The Poisson PMF: Let f (k) = 1 λke−λ k!
for k = 0, 1, 2, · · · and f (x) = 0 for other values of x. Then f is called the Poisson PMF with parameter λ.
The Mean and Variance: These are
µ = λ, σ2 = λ.
Slide 13
Example
Q: A professor hits the wrong key with probability p = .001 each time he types a letter (or symbols). What is the probability of 5 or more errors in n = 2500 letters.
A: Let X be the number of errors. Then is is approximately Poisson with
λ = 2500 × .001 = 2.5.
So, P [X = k] = 1 (2.5)ke−2.5 k!
for k = 0, 1, 2, · · · and
P [X ≥ 5] = 1 − P [X ≤ 4]
4
=1−
1 (2.5)ke−2.5
k=0 k!
= .1088
Slide 14
0.25
0.20
Graph Of f(k) With λ = 2.5
f (k) = 1 (2.5)ke−2.5 k!
0
2
4
6
8
10
k
Slide 15
Jointly Distributed Random Variables
Defs: Given a model, (Ω, P ), random variables X, Y, Z, · · · : Ω → IR
are said to be jointly distributed. An Example: If a committee of size four is selected at random from 5 Dems., 5 Inds., and 5 Reps, then
X = #Dems and Y = #Reps, are JDRVs for which
P [X = 0, Y = 0] = 5 / 15 , 44
5 2 5 15 P [X = 1, Y = 1] = 1 2 / 4 ,
5 2 15 P [X = 2, Y = 2] = 2 / 4 , and P [X = Y ] = 5 + 250 + 100 = .260.
1365
Slide 16
0.15
Poisson probability
0.10
0.05
0.00
The Joint Probability Mass Function Two RVs: If X and Y are JD discrete RVs, then their joint probability mass function is
f (x, y) = P [X = x, Y = y] for x, y ∈ IR. Several: If X1, · · · , Xm are JD discrete RVs, then their joint probability mass function is
f (x1, · · · , xm) = P [X1 = x1, · · · , Xm = xm] for x1, · · · , xm ∈ IR. Vector Notation: Let X = (X1, · · · , Xm) and
f (x) = P [X = x] for x = (x1, · · · , xm) ∈ IRm. Properties: Similar to the univariate case.
Slide 17
Partitions If n ≥ 1 and n1, · · · , nm ≥ 0 are integers for which
n1 + · · · + nm = n,
then a set of n elements may be partitioned into m subsets of sizes n1, · · · , nm in
ways.
n = n! n1, · · · , nm n1! × · · · × nm!
Example: From the letters of the word MISSISSIPPI
11
11!
4, 1, 2, 4 = 1! × 2! × 4!2 = 34650.
distinguishable lists may be formed.
Slide 18
Example
If a balanced (6-sided) die is rolled 12 times, then the probability that each face appears twice is
12! ( 1 )12 = .0034. 2!6 6 For an outcome is
ω = (i1, · · · , i12),
where 1 ≤ i1, · · · , i12 ≤ 6; there are #Ω = 612
such outcomes on
12
= 12!
2, 2, 2, 2, 2, 2 2!6
of which each face appears twice.
Slide 19
Multinomial Distributions
A Loaded Die: Now consider an m-sided, loaded die. Let
pi = P rob[i spots]
on a single role. So, p1, · · · , pm ≥ 0 and p1 + · · · + pm = 1.
Repeated Trials: Suppose that the die is rolled n times, and let Xi = #roles with i spots for i = 1, · · · , m. Then
f (x1, · · · , xm) = P [X1 = x1, · · · , Xm = xm]
=
n
px1 × · · · × pxm
x1, · · · , xm 1
m
for integers x1, · · · , xm ≥ 0 with x1 + · · · + xm = n.
Slide 20
Marginal Distributions Two Variables
Let X and Y by JD discrete RVs with joint PMF
f (x, y) = P [X = x, Y = y]
and ranges X and Y. So, f (x, y) = 0 unless x ∈ X and y ∈ Y. Then X and Y have individual (marginal) PMFs
fX (x) = f (x, y),
y∈Y
fY (y) = f (x, y).
x∈X
For {X = x} = {X = x, Y = y}
y∈Y
and, therefore,
P [X = x] = P [X = x, Y = y].
y∈Y
Slide 21
Example
Two tickets are drawn w.o.r. from a box with
1 ticket labelled one, 2 tickets labelled two, 3 ticket labelled three,
Let
X = label on first ticket, Y = label on second.
Then
Table of f (x, y)
x,y
1 2 3 fX (x)
1
0
2
3
30 30
1 6
2
2
2
6
2
30 30 30
6
3
3
6
6
3
30 30 30
6
fY (y)
1 6
2 6
3 6
Slide 22
Marginal Distributions Several Variables
Let X = (X1, · · · , Xj)
and Y = (Y1, · · · , Yk)
be JD discrete RVs with joint PMF f (x, y) = P [X = x, Y = y]
and ranges X = X(Ω) and Y = Y(Ω). Then X and Y have individual (marginal) joint PMFs
fX(x) = f (x, y)
y∈Y
and fY(y) = f (x, y).
x∈X
Slide 23
Conditional Distributions
Let X and Y have joint PMF f . If fX (x) > 0, then the conditional PMF of Y given X is
f (x, y) fY |X (y|x) = fX (x) . Thus,
P [X = x, Y = y] fY |X (y|x) = P [X = x]
= P [Y = y|X = x].
Note: fY |X is a PMF, since
1
fY |X (y|x) = fX (x) f (x, y) = 1.
y∈Y
y∈Y
Note: Can reverse the roles of X and Y .
Slide 24
Independence
JDRVs X and Y are independent if
P [X ∈ A, Y ∈ B] = P [X ∈ A]P [Y ∈ B]
for all nice subsets A, B ⊆ IR (for example, intervals).
Conditions for Independence
PMFs: If X and Y are discrete, then X and Y are independent iff
f (x, y) = fX (x)fY (y)
(∗)
for all x and y.
Example: If E and F are independent events, then 1E and 1F are independent random variables. For example,
P [1E = 1, 1F = 1] = P (E ∩ F ) = P (E)P (F ) = P [1E = 1]P [1F = 1]
Slide 25
Several Variables Jointly distributed random variables X1, · · · , Xm are independent if
P [X1 ∈ B1, · · · , Xm ∈ Bm] = P [X1 ∈ B1] × · · · P [Xm ∈ Bm]
for all Bi ⊆ IR. As in the case of two variables, this is equivalent to
f (x1, · · · , xm) = f1(xi) × · · · × fm(xm) for all x1, · · · , Xm.
Slide 26
• Random Variables • Probability Mass Functions • Expectation: The Mean and Variance • Special Distributions
Hypergeometric Binomial Poisson • Joint Distributions • Independence
Slide 1
Random Variables Consider a probability model (Ω, P ). Definition. A random variable is a function
X : Ω → IR. If X is a RV, let X = {X(ω) : ω ∈ Ω}, the range of X. Then X is discrete if X = {x1, x2, · · · }, (finte or infinite). Notation. If X is a random variable write
P [X ∈ B] = P ({ω : X(ω) ∈ B}) for B ⊆ IR. Definition. If X is a discrete RV, then the probability mass function of X is defined by
f (x) = P [X = x]. for x ∈ IR. Alternative Notation: fX .
Slide 2
Example
A committee of size n = 4 is selected from 5 men and 5 women. Then
Ω = combinations of 4, #Ω = 10 = 210, 4 P (A) = #A . #Ω
Let X = #women
Then
X = {0, 1, 2, 3, 4},
f (x) = 5 x
5 / 10 4−x 4
for x = 0, · · · , 4, and f (x) = 0 for other values of
x. Thus f (0) = f (4) = 5/210, f (1) = f (3) = 50/210, and f (2) = 100/210.
Slide 3
Hypergeometric Distributions
If a sample of size n is drawn w.o.r. from a box containing R ≥ 1 red tickets and N − R ≥ 1 white tickets, then the PMF of
X = #red tickets in the sample
is
f (r) = R r
N −R / N n−r n
for r = 0, · · · , n and f (x) = 0 for other x.
Def: Called Hypergeometric with parameters n, N , and R.
Slide 4
Properties of PMFs
If f is the PMF (probability mass function) of X,
then
f (x) ≥ 0 for all x,
(1)
f (x) = 0 unless x ∈ X ,
(2)
where X is the range of X, and
f (x) = 1.
(3)
x∈X
Moreover
P [X ∈ B] = f (x)
(4)
x∈B
for B ⊆ IR. Conversely, any function f that
satisfies (1), (2), and (3) is the PMF of some
random variable X.
Notes a). Henceforth ”PMF” means any function that satisfies (1), (2), and (3).
b). We can specify a model by giving X and f , subject to the first three conditions.
Slide 5
Expectation
If X is a discrete random variable with PMF f and randge X , then the expectation of X is
E(X) = xf (x),
x∈X
provided that the sum converges absolutely.
Example: Committees. In the committees example
E(X) = 0 × 5 + 1 × 50 + 2 × 100
210
210
210
+ 3 × 50 + 4 × 5
210
210
= 2.
Slide 6
Indicator Variables If A is an event, then the indicator function of A is
1 if ω ∈ A 1A(ω) = 0 otherwise.
The range is {0, 1}, f (0) = P (Ac) = 1 − P (A), f (1) = P (A),
and E(1A) = 0 × P (Ac) + 1 × P (A) = P (A).
Note: The language of RVs subsumes that of events.
Slide 7
Transformations . If X is a discrete random variable and Y = w(X), then Y has PMF
fY (y) =
fX (x).
x:w(x)=y
For example, P [X2 = y] = f (√y) + f (−√y)
for y > 0.
The Transformation Formula. If X is a discrete random variable and Y = w(X)
E(Y ) = w(x)fX(x),
(5)
x∈X
provided that the sum converges absolutely.
Slide 8
The Mean and Variance
If X has PMF f , then
E(X) = xf (x)
x∈X
depends only on f , assuming that it exists. It is also called the mean of f and/or X and denoted by µ. Thus. µ = E(X).
The variance of X is defined by
σ2 = E[(X − µ)2].
Thus
σ2 = (x − µ)2f (x).
x∈X
which depends only√of f ; and σ2 is also called the variance of f . σ = σ2 is called the standard
deviation of X and/or f.
A Useful Formula: If X is discrete with mean µ and variance σ2, then
E[X2] = σ2 + µ2
Slide 9
Binomial Distributions Independent Trials: A basic experiment with Ω0 is repeated n times. Events dependending on different trials (replications) are independent.
Examplea). Sampling with replacement. b). Gamling–e.g. coins tossing and roulette.
Basic Question. Given an event A0 ⊆ Ω0, what is the probability that A0 occurs k times in n trials? Formally, let
Ai = {A0 occurs on the ith trial}, X = 1A1 + · · · + 1An.
Answer P [X = k] = n pkqn−k k
for k = 0, · · · , n, where q = 1 − p. Def: Called Binomial with parameters n and p. The Mean and Variance: µ = np and σ2 = npq.
Slide 10
0.25
0.20
Graph Of f(k) With n = 10 and p = .5
f (k) = 10 ( 1 )10 k2
0
2
4
6
8
10
k
Slide 11
Example
Q: What is the probability that South gets no aces on at least k = 5 or n = 9 hands?
A: Let
Ai = {no aces on the ithhand}, X = 1A1 + · · · + 1A9.
Then
48
P (Ai) =
13 52
13
for i = 1, · · · , 9. So,
= .3038 = p, say
P [X = k] = 9 pk(1 − p)n−k k
for k = 0, · · · , 9, and
9
P [X ≥ 5] =
9 pk(1 − p)n−k.
k=5 k
So, P [X ≥ 5] = .1035.
Slide 12
0.15
Binomial probability
0.10
0.05
0.00
Poisson Distributions AKA The Law of Rare Events
The Derivation: As
n → ∞,
p → 0,
0 < λ = np < ∞ constant,
lim n pk(1 − p)n−k = 1 λke−λ
k
k!
The Poisson PMF: Let f (k) = 1 λke−λ k!
for k = 0, 1, 2, · · · and f (x) = 0 for other values of x. Then f is called the Poisson PMF with parameter λ.
The Mean and Variance: These are
µ = λ, σ2 = λ.
Slide 13
Example
Q: A professor hits the wrong key with probability p = .001 each time he types a letter (or symbols). What is the probability of 5 or more errors in n = 2500 letters.
A: Let X be the number of errors. Then is is approximately Poisson with
λ = 2500 × .001 = 2.5.
So, P [X = k] = 1 (2.5)ke−2.5 k!
for k = 0, 1, 2, · · · and
P [X ≥ 5] = 1 − P [X ≤ 4]
4
=1−
1 (2.5)ke−2.5
k=0 k!
= .1088
Slide 14
0.25
0.20
Graph Of f(k) With λ = 2.5
f (k) = 1 (2.5)ke−2.5 k!
0
2
4
6
8
10
k
Slide 15
Jointly Distributed Random Variables
Defs: Given a model, (Ω, P ), random variables X, Y, Z, · · · : Ω → IR
are said to be jointly distributed. An Example: If a committee of size four is selected at random from 5 Dems., 5 Inds., and 5 Reps, then
X = #Dems and Y = #Reps, are JDRVs for which
P [X = 0, Y = 0] = 5 / 15 , 44
5 2 5 15 P [X = 1, Y = 1] = 1 2 / 4 ,
5 2 15 P [X = 2, Y = 2] = 2 / 4 , and P [X = Y ] = 5 + 250 + 100 = .260.
1365
Slide 16
0.15
Poisson probability
0.10
0.05
0.00
The Joint Probability Mass Function Two RVs: If X and Y are JD discrete RVs, then their joint probability mass function is
f (x, y) = P [X = x, Y = y] for x, y ∈ IR. Several: If X1, · · · , Xm are JD discrete RVs, then their joint probability mass function is
f (x1, · · · , xm) = P [X1 = x1, · · · , Xm = xm] for x1, · · · , xm ∈ IR. Vector Notation: Let X = (X1, · · · , Xm) and
f (x) = P [X = x] for x = (x1, · · · , xm) ∈ IRm. Properties: Similar to the univariate case.
Slide 17
Partitions If n ≥ 1 and n1, · · · , nm ≥ 0 are integers for which
n1 + · · · + nm = n,
then a set of n elements may be partitioned into m subsets of sizes n1, · · · , nm in
ways.
n = n! n1, · · · , nm n1! × · · · × nm!
Example: From the letters of the word MISSISSIPPI
11
11!
4, 1, 2, 4 = 1! × 2! × 4!2 = 34650.
distinguishable lists may be formed.
Slide 18
Example
If a balanced (6-sided) die is rolled 12 times, then the probability that each face appears twice is
12! ( 1 )12 = .0034. 2!6 6 For an outcome is
ω = (i1, · · · , i12),
where 1 ≤ i1, · · · , i12 ≤ 6; there are #Ω = 612
such outcomes on
12
= 12!
2, 2, 2, 2, 2, 2 2!6
of which each face appears twice.
Slide 19
Multinomial Distributions
A Loaded Die: Now consider an m-sided, loaded die. Let
pi = P rob[i spots]
on a single role. So, p1, · · · , pm ≥ 0 and p1 + · · · + pm = 1.
Repeated Trials: Suppose that the die is rolled n times, and let Xi = #roles with i spots for i = 1, · · · , m. Then
f (x1, · · · , xm) = P [X1 = x1, · · · , Xm = xm]
=
n
px1 × · · · × pxm
x1, · · · , xm 1
m
for integers x1, · · · , xm ≥ 0 with x1 + · · · + xm = n.
Slide 20
Marginal Distributions Two Variables
Let X and Y by JD discrete RVs with joint PMF
f (x, y) = P [X = x, Y = y]
and ranges X and Y. So, f (x, y) = 0 unless x ∈ X and y ∈ Y. Then X and Y have individual (marginal) PMFs
fX (x) = f (x, y),
y∈Y
fY (y) = f (x, y).
x∈X
For {X = x} = {X = x, Y = y}
y∈Y
and, therefore,
P [X = x] = P [X = x, Y = y].
y∈Y
Slide 21
Example
Two tickets are drawn w.o.r. from a box with
1 ticket labelled one, 2 tickets labelled two, 3 ticket labelled three,
Let
X = label on first ticket, Y = label on second.
Then
Table of f (x, y)
x,y
1 2 3 fX (x)
1
0
2
3
30 30
1 6
2
2
2
6
2
30 30 30
6
3
3
6
6
3
30 30 30
6
fY (y)
1 6
2 6
3 6
Slide 22
Marginal Distributions Several Variables
Let X = (X1, · · · , Xj)
and Y = (Y1, · · · , Yk)
be JD discrete RVs with joint PMF f (x, y) = P [X = x, Y = y]
and ranges X = X(Ω) and Y = Y(Ω). Then X and Y have individual (marginal) joint PMFs
fX(x) = f (x, y)
y∈Y
and fY(y) = f (x, y).
x∈X
Slide 23
Conditional Distributions
Let X and Y have joint PMF f . If fX (x) > 0, then the conditional PMF of Y given X is
f (x, y) fY |X (y|x) = fX (x) . Thus,
P [X = x, Y = y] fY |X (y|x) = P [X = x]
= P [Y = y|X = x].
Note: fY |X is a PMF, since
1
fY |X (y|x) = fX (x) f (x, y) = 1.
y∈Y
y∈Y
Note: Can reverse the roles of X and Y .
Slide 24
Independence
JDRVs X and Y are independent if
P [X ∈ A, Y ∈ B] = P [X ∈ A]P [Y ∈ B]
for all nice subsets A, B ⊆ IR (for example, intervals).
Conditions for Independence
PMFs: If X and Y are discrete, then X and Y are independent iff
f (x, y) = fX (x)fY (y)
(∗)
for all x and y.
Example: If E and F are independent events, then 1E and 1F are independent random variables. For example,
P [1E = 1, 1F = 1] = P (E ∩ F ) = P (E)P (F ) = P [1E = 1]P [1F = 1]
Slide 25
Several Variables Jointly distributed random variables X1, · · · , Xm are independent if
P [X1 ∈ B1, · · · , Xm ∈ Bm] = P [X1 ∈ B1] × · · · P [Xm ∈ Bm]
for all Bi ⊆ IR. As in the case of two variables, this is equivalent to
f (x1, · · · , xm) = f1(xi) × · · · × fm(xm) for all x1, · · · , Xm.
Slide 26