# Discrete Random Variables Chs. 2, 3, 4 Random Variables

## Transcript Of Discrete Random Variables Chs. 2, 3, 4 Random Variables

Discrete Random Variables Chs. 2, 3, 4

• Random Variables • Probability Mass Functions • Expectation: The Mean and Variance • Special Distributions

Hypergeometric Binomial Poisson • Joint Distributions • Independence

Slide 1

Random Variables Consider a probability model (Ω, P ). Deﬁnition. A random variable is a function

X : Ω → IR. If X is a RV, let X = {X(ω) : ω ∈ Ω}, the range of X. Then X is discrete if X = {x1, x2, · · · }, (ﬁnte or inﬁnite). Notation. If X is a random variable write

P [X ∈ B] = P ({ω : X(ω) ∈ B}) for B ⊆ IR. Deﬁnition. If X is a discrete RV, then the probability mass function of X is deﬁned by

f (x) = P [X = x]. for x ∈ IR. Alternative Notation: fX .

Slide 2

Example

A committee of size n = 4 is selected from 5 men and 5 women. Then

Ω = combinations of 4, #Ω = 10 = 210, 4 P (A) = #A . #Ω

Let X = #women

Then

X = {0, 1, 2, 3, 4},

f (x) = 5 x

5 / 10 4−x 4

for x = 0, · · · , 4, and f (x) = 0 for other values of

x. Thus f (0) = f (4) = 5/210, f (1) = f (3) = 50/210, and f (2) = 100/210.

Slide 3

Hypergeometric Distributions

If a sample of size n is drawn w.o.r. from a box containing R ≥ 1 red tickets and N − R ≥ 1 white tickets, then the PMF of

X = #red tickets in the sample

is

f (r) = R r

N −R / N n−r n

for r = 0, · · · , n and f (x) = 0 for other x.

Def: Called Hypergeometric with parameters n, N , and R.

Slide 4

Properties of PMFs

If f is the PMF (probability mass function) of X,

then

f (x) ≥ 0 for all x,

(1)

f (x) = 0 unless x ∈ X ,

(2)

where X is the range of X, and

f (x) = 1.

(3)

x∈X

Moreover

P [X ∈ B] = f (x)

(4)

x∈B

for B ⊆ IR. Conversely, any function f that

satisﬁes (1), (2), and (3) is the PMF of some

random variable X.

Notes a). Henceforth ”PMF” means any function that satisﬁes (1), (2), and (3).

b). We can specify a model by giving X and f , subject to the ﬁrst three conditions.

Slide 5

Expectation

If X is a discrete random variable with PMF f and randge X , then the expectation of X is

E(X) = xf (x),

x∈X

provided that the sum converges absolutely.

Example: Committees. In the committees example

E(X) = 0 × 5 + 1 × 50 + 2 × 100

210

210

210

+ 3 × 50 + 4 × 5

210

210

= 2.

Slide 6

Indicator Variables If A is an event, then the indicator function of A is

1 if ω ∈ A 1A(ω) = 0 otherwise.

The range is {0, 1}, f (0) = P (Ac) = 1 − P (A), f (1) = P (A),

and E(1A) = 0 × P (Ac) + 1 × P (A) = P (A).

Note: The language of RVs subsumes that of events.

Slide 7

Transformations . If X is a discrete random variable and Y = w(X), then Y has PMF

fY (y) =

fX (x).

x:w(x)=y

For example, P [X2 = y] = f (√y) + f (−√y)

for y > 0.

The Transformation Formula. If X is a discrete random variable and Y = w(X)

E(Y ) = w(x)fX(x),

(5)

x∈X

provided that the sum converges absolutely.

Slide 8

The Mean and Variance

If X has PMF f , then

E(X) = xf (x)

x∈X

depends only on f , assuming that it exists. It is also called the mean of f and/or X and denoted by µ. Thus. µ = E(X).

The variance of X is deﬁned by

σ2 = E[(X − µ)2].

Thus

σ2 = (x − µ)2f (x).

x∈X

which depends only√of f ; and σ2 is also called the variance of f . σ = σ2 is called the standard

deviation of X and/or f.

A Useful Formula: If X is discrete with mean µ and variance σ2, then

E[X2] = σ2 + µ2

Slide 9

Binomial Distributions Independent Trials: A basic experiment with Ω0 is repeated n times. Events dependending on diﬀerent trials (replications) are independent.

Examplea). Sampling with replacement. b). Gamling–e.g. coins tossing and roulette.

Basic Question. Given an event A0 ⊆ Ω0, what is the probability that A0 occurs k times in n trials? Formally, let

Ai = {A0 occurs on the ith trial}, X = 1A1 + · · · + 1An.

Answer P [X = k] = n pkqn−k k

for k = 0, · · · , n, where q = 1 − p. Def: Called Binomial with parameters n and p. The Mean and Variance: µ = np and σ2 = npq.

Slide 10

0.25

0.20

Graph Of f(k) With n = 10 and p = .5

f (k) = 10 ( 1 )10 k2

0

2

4

6

8

10

k

Slide 11

Example

Q: What is the probability that South gets no aces on at least k = 5 or n = 9 hands?

A: Let

Ai = {no aces on the ithhand}, X = 1A1 + · · · + 1A9.

Then

48

P (Ai) =

13 52

13

for i = 1, · · · , 9. So,

= .3038 = p, say

P [X = k] = 9 pk(1 − p)n−k k

for k = 0, · · · , 9, and

9

P [X ≥ 5] =

9 pk(1 − p)n−k.

k=5 k

So, P [X ≥ 5] = .1035.

Slide 12

0.15

Binomial probability

0.10

0.05

0.00

Poisson Distributions AKA The Law of Rare Events

The Derivation: As

n → ∞,

p → 0,

0 < λ = np < ∞ constant,

lim n pk(1 − p)n−k = 1 λke−λ

k

k!

The Poisson PMF: Let f (k) = 1 λke−λ k!

for k = 0, 1, 2, · · · and f (x) = 0 for other values of x. Then f is called the Poisson PMF with parameter λ.

The Mean and Variance: These are

µ = λ, σ2 = λ.

Slide 13

Example

Q: A professor hits the wrong key with probability p = .001 each time he types a letter (or symbols). What is the probability of 5 or more errors in n = 2500 letters.

A: Let X be the number of errors. Then is is approximately Poisson with

λ = 2500 × .001 = 2.5.

So, P [X = k] = 1 (2.5)ke−2.5 k!

for k = 0, 1, 2, · · · and

P [X ≥ 5] = 1 − P [X ≤ 4]

4

=1−

1 (2.5)ke−2.5

k=0 k!

= .1088

Slide 14

0.25

0.20

Graph Of f(k) With λ = 2.5

f (k) = 1 (2.5)ke−2.5 k!

0

2

4

6

8

10

k

Slide 15

Jointly Distributed Random Variables

Defs: Given a model, (Ω, P ), random variables X, Y, Z, · · · : Ω → IR

are said to be jointly distributed. An Example: If a committee of size four is selected at random from 5 Dems., 5 Inds., and 5 Reps, then

X = #Dems and Y = #Reps, are JDRVs for which

P [X = 0, Y = 0] = 5 / 15 , 44

5 2 5 15 P [X = 1, Y = 1] = 1 2 / 4 ,

5 2 15 P [X = 2, Y = 2] = 2 / 4 , and P [X = Y ] = 5 + 250 + 100 = .260.

1365

Slide 16

0.15

Poisson probability

0.10

0.05

0.00

The Joint Probability Mass Function Two RVs: If X and Y are JD discrete RVs, then their joint probability mass function is

f (x, y) = P [X = x, Y = y] for x, y ∈ IR. Several: If X1, · · · , Xm are JD discrete RVs, then their joint probability mass function is

f (x1, · · · , xm) = P [X1 = x1, · · · , Xm = xm] for x1, · · · , xm ∈ IR. Vector Notation: Let X = (X1, · · · , Xm) and

f (x) = P [X = x] for x = (x1, · · · , xm) ∈ IRm. Properties: Similar to the univariate case.

Slide 17

Partitions If n ≥ 1 and n1, · · · , nm ≥ 0 are integers for which

n1 + · · · + nm = n,

then a set of n elements may be partitioned into m subsets of sizes n1, · · · , nm in

ways.

n = n! n1, · · · , nm n1! × · · · × nm!

Example: From the letters of the word MISSISSIPPI

11

11!

4, 1, 2, 4 = 1! × 2! × 4!2 = 34650.

distinguishable lists may be formed.

Slide 18

Example

If a balanced (6-sided) die is rolled 12 times, then the probability that each face appears twice is

12! ( 1 )12 = .0034. 2!6 6 For an outcome is

ω = (i1, · · · , i12),

where 1 ≤ i1, · · · , i12 ≤ 6; there are #Ω = 612

such outcomes on

12

= 12!

2, 2, 2, 2, 2, 2 2!6

of which each face appears twice.

Slide 19

Multinomial Distributions

A Loaded Die: Now consider an m-sided, loaded die. Let

pi = P rob[i spots]

on a single role. So, p1, · · · , pm ≥ 0 and p1 + · · · + pm = 1.

Repeated Trials: Suppose that the die is rolled n times, and let Xi = #roles with i spots for i = 1, · · · , m. Then

f (x1, · · · , xm) = P [X1 = x1, · · · , Xm = xm]

=

n

px1 × · · · × pxm

x1, · · · , xm 1

m

for integers x1, · · · , xm ≥ 0 with x1 + · · · + xm = n.

Slide 20

Marginal Distributions Two Variables

Let X and Y by JD discrete RVs with joint PMF

f (x, y) = P [X = x, Y = y]

and ranges X and Y. So, f (x, y) = 0 unless x ∈ X and y ∈ Y. Then X and Y have individual (marginal) PMFs

fX (x) = f (x, y),

y∈Y

fY (y) = f (x, y).

x∈X

For {X = x} = {X = x, Y = y}

y∈Y

and, therefore,

P [X = x] = P [X = x, Y = y].

y∈Y

Slide 21

Example

Two tickets are drawn w.o.r. from a box with

1 ticket labelled one, 2 tickets labelled two, 3 ticket labelled three,

Let

X = label on ﬁrst ticket, Y = label on second.

Then

Table of f (x, y)

x,y

1 2 3 fX (x)

1

0

2

3

30 30

1 6

2

2

2

6

2

30 30 30

6

3

3

6

6

3

30 30 30

6

fY (y)

1 6

2 6

3 6

Slide 22

Marginal Distributions Several Variables

Let X = (X1, · · · , Xj)

and Y = (Y1, · · · , Yk)

be JD discrete RVs with joint PMF f (x, y) = P [X = x, Y = y]

and ranges X = X(Ω) and Y = Y(Ω). Then X and Y have individual (marginal) joint PMFs

fX(x) = f (x, y)

y∈Y

and fY(y) = f (x, y).

x∈X

Slide 23

Conditional Distributions

Let X and Y have joint PMF f . If fX (x) > 0, then the conditional PMF of Y given X is

f (x, y) fY |X (y|x) = fX (x) . Thus,

P [X = x, Y = y] fY |X (y|x) = P [X = x]

= P [Y = y|X = x].

Note: fY |X is a PMF, since

1

fY |X (y|x) = fX (x) f (x, y) = 1.

y∈Y

y∈Y

Note: Can reverse the roles of X and Y .

Slide 24

Independence

JDRVs X and Y are independent if

P [X ∈ A, Y ∈ B] = P [X ∈ A]P [Y ∈ B]

for all nice subsets A, B ⊆ IR (for example, intervals).

Conditions for Independence

PMFs: If X and Y are discrete, then X and Y are independent iﬀ

f (x, y) = fX (x)fY (y)

(∗)

for all x and y.

Example: If E and F are independent events, then 1E and 1F are independent random variables. For example,

P [1E = 1, 1F = 1] = P (E ∩ F ) = P (E)P (F ) = P [1E = 1]P [1F = 1]

Slide 25

Several Variables Jointly distributed random variables X1, · · · , Xm are independent if

P [X1 ∈ B1, · · · , Xm ∈ Bm] = P [X1 ∈ B1] × · · · P [Xm ∈ Bm]

for all Bi ⊆ IR. As in the case of two variables, this is equivalent to

f (x1, · · · , xm) = f1(xi) × · · · × fm(xm) for all x1, · · · , Xm.

Slide 26

• Random Variables • Probability Mass Functions • Expectation: The Mean and Variance • Special Distributions

Hypergeometric Binomial Poisson • Joint Distributions • Independence

Slide 1

Random Variables Consider a probability model (Ω, P ). Deﬁnition. A random variable is a function

X : Ω → IR. If X is a RV, let X = {X(ω) : ω ∈ Ω}, the range of X. Then X is discrete if X = {x1, x2, · · · }, (ﬁnte or inﬁnite). Notation. If X is a random variable write

P [X ∈ B] = P ({ω : X(ω) ∈ B}) for B ⊆ IR. Deﬁnition. If X is a discrete RV, then the probability mass function of X is deﬁned by

f (x) = P [X = x]. for x ∈ IR. Alternative Notation: fX .

Slide 2

Example

A committee of size n = 4 is selected from 5 men and 5 women. Then

Ω = combinations of 4, #Ω = 10 = 210, 4 P (A) = #A . #Ω

Let X = #women

Then

X = {0, 1, 2, 3, 4},

f (x) = 5 x

5 / 10 4−x 4

for x = 0, · · · , 4, and f (x) = 0 for other values of

x. Thus f (0) = f (4) = 5/210, f (1) = f (3) = 50/210, and f (2) = 100/210.

Slide 3

Hypergeometric Distributions

If a sample of size n is drawn w.o.r. from a box containing R ≥ 1 red tickets and N − R ≥ 1 white tickets, then the PMF of

X = #red tickets in the sample

is

f (r) = R r

N −R / N n−r n

for r = 0, · · · , n and f (x) = 0 for other x.

Def: Called Hypergeometric with parameters n, N , and R.

Slide 4

Properties of PMFs

If f is the PMF (probability mass function) of X,

then

f (x) ≥ 0 for all x,

(1)

f (x) = 0 unless x ∈ X ,

(2)

where X is the range of X, and

f (x) = 1.

(3)

x∈X

Moreover

P [X ∈ B] = f (x)

(4)

x∈B

for B ⊆ IR. Conversely, any function f that

satisﬁes (1), (2), and (3) is the PMF of some

random variable X.

Notes a). Henceforth ”PMF” means any function that satisﬁes (1), (2), and (3).

b). We can specify a model by giving X and f , subject to the ﬁrst three conditions.

Slide 5

Expectation

If X is a discrete random variable with PMF f and randge X , then the expectation of X is

E(X) = xf (x),

x∈X

provided that the sum converges absolutely.

Example: Committees. In the committees example

E(X) = 0 × 5 + 1 × 50 + 2 × 100

210

210

210

+ 3 × 50 + 4 × 5

210

210

= 2.

Slide 6

Indicator Variables If A is an event, then the indicator function of A is

1 if ω ∈ A 1A(ω) = 0 otherwise.

The range is {0, 1}, f (0) = P (Ac) = 1 − P (A), f (1) = P (A),

and E(1A) = 0 × P (Ac) + 1 × P (A) = P (A).

Note: The language of RVs subsumes that of events.

Slide 7

Transformations . If X is a discrete random variable and Y = w(X), then Y has PMF

fY (y) =

fX (x).

x:w(x)=y

For example, P [X2 = y] = f (√y) + f (−√y)

for y > 0.

The Transformation Formula. If X is a discrete random variable and Y = w(X)

E(Y ) = w(x)fX(x),

(5)

x∈X

provided that the sum converges absolutely.

Slide 8

The Mean and Variance

If X has PMF f , then

E(X) = xf (x)

x∈X

depends only on f , assuming that it exists. It is also called the mean of f and/or X and denoted by µ. Thus. µ = E(X).

The variance of X is deﬁned by

σ2 = E[(X − µ)2].

Thus

σ2 = (x − µ)2f (x).

x∈X

which depends only√of f ; and σ2 is also called the variance of f . σ = σ2 is called the standard

deviation of X and/or f.

A Useful Formula: If X is discrete with mean µ and variance σ2, then

E[X2] = σ2 + µ2

Slide 9

Binomial Distributions Independent Trials: A basic experiment with Ω0 is repeated n times. Events dependending on diﬀerent trials (replications) are independent.

Examplea). Sampling with replacement. b). Gamling–e.g. coins tossing and roulette.

Basic Question. Given an event A0 ⊆ Ω0, what is the probability that A0 occurs k times in n trials? Formally, let

Ai = {A0 occurs on the ith trial}, X = 1A1 + · · · + 1An.

Answer P [X = k] = n pkqn−k k

for k = 0, · · · , n, where q = 1 − p. Def: Called Binomial with parameters n and p. The Mean and Variance: µ = np and σ2 = npq.

Slide 10

0.25

0.20

Graph Of f(k) With n = 10 and p = .5

f (k) = 10 ( 1 )10 k2

0

2

4

6

8

10

k

Slide 11

Example

Q: What is the probability that South gets no aces on at least k = 5 or n = 9 hands?

A: Let

Ai = {no aces on the ithhand}, X = 1A1 + · · · + 1A9.

Then

48

P (Ai) =

13 52

13

for i = 1, · · · , 9. So,

= .3038 = p, say

P [X = k] = 9 pk(1 − p)n−k k

for k = 0, · · · , 9, and

9

P [X ≥ 5] =

9 pk(1 − p)n−k.

k=5 k

So, P [X ≥ 5] = .1035.

Slide 12

0.15

Binomial probability

0.10

0.05

0.00

Poisson Distributions AKA The Law of Rare Events

The Derivation: As

n → ∞,

p → 0,

0 < λ = np < ∞ constant,

lim n pk(1 − p)n−k = 1 λke−λ

k

k!

The Poisson PMF: Let f (k) = 1 λke−λ k!

for k = 0, 1, 2, · · · and f (x) = 0 for other values of x. Then f is called the Poisson PMF with parameter λ.

The Mean and Variance: These are

µ = λ, σ2 = λ.

Slide 13

Example

Q: A professor hits the wrong key with probability p = .001 each time he types a letter (or symbols). What is the probability of 5 or more errors in n = 2500 letters.

A: Let X be the number of errors. Then is is approximately Poisson with

λ = 2500 × .001 = 2.5.

So, P [X = k] = 1 (2.5)ke−2.5 k!

for k = 0, 1, 2, · · · and

P [X ≥ 5] = 1 − P [X ≤ 4]

4

=1−

1 (2.5)ke−2.5

k=0 k!

= .1088

Slide 14

0.25

0.20

Graph Of f(k) With λ = 2.5

f (k) = 1 (2.5)ke−2.5 k!

0

2

4

6

8

10

k

Slide 15

Jointly Distributed Random Variables

Defs: Given a model, (Ω, P ), random variables X, Y, Z, · · · : Ω → IR

are said to be jointly distributed. An Example: If a committee of size four is selected at random from 5 Dems., 5 Inds., and 5 Reps, then

X = #Dems and Y = #Reps, are JDRVs for which

P [X = 0, Y = 0] = 5 / 15 , 44

5 2 5 15 P [X = 1, Y = 1] = 1 2 / 4 ,

5 2 15 P [X = 2, Y = 2] = 2 / 4 , and P [X = Y ] = 5 + 250 + 100 = .260.

1365

Slide 16

0.15

Poisson probability

0.10

0.05

0.00

The Joint Probability Mass Function Two RVs: If X and Y are JD discrete RVs, then their joint probability mass function is

f (x, y) = P [X = x, Y = y] for x, y ∈ IR. Several: If X1, · · · , Xm are JD discrete RVs, then their joint probability mass function is

f (x1, · · · , xm) = P [X1 = x1, · · · , Xm = xm] for x1, · · · , xm ∈ IR. Vector Notation: Let X = (X1, · · · , Xm) and

f (x) = P [X = x] for x = (x1, · · · , xm) ∈ IRm. Properties: Similar to the univariate case.

Slide 17

Partitions If n ≥ 1 and n1, · · · , nm ≥ 0 are integers for which

n1 + · · · + nm = n,

then a set of n elements may be partitioned into m subsets of sizes n1, · · · , nm in

ways.

n = n! n1, · · · , nm n1! × · · · × nm!

Example: From the letters of the word MISSISSIPPI

11

11!

4, 1, 2, 4 = 1! × 2! × 4!2 = 34650.

distinguishable lists may be formed.

Slide 18

Example

If a balanced (6-sided) die is rolled 12 times, then the probability that each face appears twice is

12! ( 1 )12 = .0034. 2!6 6 For an outcome is

ω = (i1, · · · , i12),

where 1 ≤ i1, · · · , i12 ≤ 6; there are #Ω = 612

such outcomes on

12

= 12!

2, 2, 2, 2, 2, 2 2!6

of which each face appears twice.

Slide 19

Multinomial Distributions

A Loaded Die: Now consider an m-sided, loaded die. Let

pi = P rob[i spots]

on a single role. So, p1, · · · , pm ≥ 0 and p1 + · · · + pm = 1.

Repeated Trials: Suppose that the die is rolled n times, and let Xi = #roles with i spots for i = 1, · · · , m. Then

f (x1, · · · , xm) = P [X1 = x1, · · · , Xm = xm]

=

n

px1 × · · · × pxm

x1, · · · , xm 1

m

for integers x1, · · · , xm ≥ 0 with x1 + · · · + xm = n.

Slide 20

Marginal Distributions Two Variables

Let X and Y by JD discrete RVs with joint PMF

f (x, y) = P [X = x, Y = y]

and ranges X and Y. So, f (x, y) = 0 unless x ∈ X and y ∈ Y. Then X and Y have individual (marginal) PMFs

fX (x) = f (x, y),

y∈Y

fY (y) = f (x, y).

x∈X

For {X = x} = {X = x, Y = y}

y∈Y

and, therefore,

P [X = x] = P [X = x, Y = y].

y∈Y

Slide 21

Example

Two tickets are drawn w.o.r. from a box with

1 ticket labelled one, 2 tickets labelled two, 3 ticket labelled three,

Let

X = label on ﬁrst ticket, Y = label on second.

Then

Table of f (x, y)

x,y

1 2 3 fX (x)

1

0

2

3

30 30

1 6

2

2

2

6

2

30 30 30

6

3

3

6

6

3

30 30 30

6

fY (y)

1 6

2 6

3 6

Slide 22

Marginal Distributions Several Variables

Let X = (X1, · · · , Xj)

and Y = (Y1, · · · , Yk)

be JD discrete RVs with joint PMF f (x, y) = P [X = x, Y = y]

and ranges X = X(Ω) and Y = Y(Ω). Then X and Y have individual (marginal) joint PMFs

fX(x) = f (x, y)

y∈Y

and fY(y) = f (x, y).

x∈X

Slide 23

Conditional Distributions

Let X and Y have joint PMF f . If fX (x) > 0, then the conditional PMF of Y given X is

f (x, y) fY |X (y|x) = fX (x) . Thus,

P [X = x, Y = y] fY |X (y|x) = P [X = x]

= P [Y = y|X = x].

Note: fY |X is a PMF, since

1

fY |X (y|x) = fX (x) f (x, y) = 1.

y∈Y

y∈Y

Note: Can reverse the roles of X and Y .

Slide 24

Independence

JDRVs X and Y are independent if

P [X ∈ A, Y ∈ B] = P [X ∈ A]P [Y ∈ B]

for all nice subsets A, B ⊆ IR (for example, intervals).

Conditions for Independence

PMFs: If X and Y are discrete, then X and Y are independent iﬀ

f (x, y) = fX (x)fY (y)

(∗)

for all x and y.

Example: If E and F are independent events, then 1E and 1F are independent random variables. For example,

P [1E = 1, 1F = 1] = P (E ∩ F ) = P (E)P (F ) = P [1E = 1]P [1F = 1]

Slide 25

Several Variables Jointly distributed random variables X1, · · · , Xm are independent if

P [X1 ∈ B1, · · · , Xm ∈ Bm] = P [X1 ∈ B1] × · · · P [Xm ∈ Bm]

for all Bi ⊆ IR. As in the case of two variables, this is equivalent to

f (x1, · · · , xm) = f1(xi) × · · · × fm(xm) for all x1, · · · , Xm.

Slide 26