One-Way Analysis of Variance F-Tests using Effect Size

Preparing to load PDF file. please wait...

0 of 0
100%
One-Way Analysis of Variance F-Tests using Effect Size

Transcript Of One-Way Analysis of Variance F-Tests using Effect Size

PASS Sample Size Software

NCSS.com

Chapter 597
One-Way Analysis of Variance F-Tests using Effect Size
Introduction
A common task in research is to compare the averages of two or more populations (groups). We might want to compare the income level of two regions, the nitrogen content of three lakes, or the effectiveness of four drugs. The one-way analysis of variance compares the means of two or more groups to determine if at least one mean is different from the others. The F test is used to determine statistical significance. F tests are non-directional in that the null hypothesis specifies that all means are equal, and the alternative hypothesis simply states that at least one mean is different from the rest. The methods described here are usually applied to the one-way experimental design. This design is an extension of the design used for the two-sample t test. Instead of two groups, there are three or more groups. In our more advanced one-way ANOVA procedure, you are required to enter hypothesized means and variances. This simplified procedure only requires the input of an effect size, usually f, as proposed by Cohen (1988).
Assumptions
Using the F test requires certain assumptions. One reason for the popularity of the F test is its robustness in the face of assumption violation. However, if an assumption is not even approximately met, the significance levels and the power of the F test are invalidated. Unfortunately, in practice it often happens that several assumptions are not met. This makes matters even worse. Hence, steps should be taken to check the assumptions before important decisions are made. The assumptions of the one-way analysis of variance are:
1. The data are continuous (not discrete). 2. The data follow the normal probability distribution. Each group is normally distributed about the
group mean. 3. The variances within the groups are equal. 4. The groups are independent. There is no relationship among the individuals in one group as
compared to another. 5. Each group is a simple random sample from its population. Each individual in the population has an
equal probability of being selected in the sample.

597-1

Β© NCSS, LLC. All Rights Reserved.

PASS Sample Size Software

One-Way Analysis of Variance F-Tests using Effect Size

NCSS.com

Technical Details for the One-Way ANOVA
Suppose G groups each have a normal distribution and equal means(πœ‡πœ‡1 = πœ‡πœ‡2 = β‹― = πœ‡πœ‡πΊπΊ). Let 𝑛𝑛1 = β‹― = 𝑛𝑛𝐺𝐺 denote the number of subjects in each group and let N denote the total sample size of all groups. Let πœ‡πœ‡Μ…π‘€π‘€ denote the weighted mean of all groups. That is
𝐺𝐺
𝑛𝑛𝑖𝑖 πœ‡πœ‡Μ…π‘€π‘€ = οΏ½ �𝑁𝑁 οΏ½ πœ‡πœ‡π‘–π‘–
𝑖𝑖=1
Let 𝜎𝜎 denote the common standard deviation of all groups.
Given the above terminology, the ratio of the mean square between groups to the mean square within groups follows a central F distribution with two parameters matching the degrees of freedom of the numerator mean square and the denominator mean square. When the null hypothesis of mean equality is rejected, the above ratio has a noncentral F distribution which also depends on the noncentrality parameter, πœ†πœ†. This parameter is calculated as
πœŽπœŽπ‘š2π‘š πœ†πœ† = 𝑁𝑁 𝜎𝜎2 where
πœŽπœŽπ‘š2π‘š = βˆ‘πΊπ‘–π‘–πΊ=1 �𝑛𝑁𝑛𝑁𝑖𝑖� (πœ‡πœ‡π‘–π‘– βˆ’ πœ‡πœ‡Μ…π‘€π‘€)2
Cohen (1988) proposed a further substitution of an effect size parameter f which is defined as

𝑓𝑓 = οΏ½πœŽπœŽπ‘š2π‘š 𝜎𝜎 2

Using this parameter, the noncentrality parameter simplifies to

πœ†πœ† = 𝑁𝑁𝑓𝑓2

Cohen (1988, 285-287) proposed the following interpretation of f: f = 0.1 is a small effect, f = 0.25 is a medium effect, and f = 0.4 is a large effect.

Cohen (1988) also referenced another effect size parameter which he named πœ‚πœ‚2 (eta-squared). This parameter is defined as

πœ‚πœ‚2 =

πœŽπœŽπ‘š2π‘š

𝑓𝑓 2 =

πœŽπœŽπ‘š2π‘š + 𝜎𝜎2 1 + 𝑓𝑓2

πœ‚πœ‚2 is the proportion of the total variation in the dependent variable that is attributable to the groups. The above equation can be solved for f as follows

𝑓𝑓2 = πœ‚πœ‚2 1 βˆ’ πœ‚πœ‚2

597-2

Β© NCSS, LLC. All Rights Reserved.

PASS Sample Size Software

One-Way Analysis of Variance F-Tests using Effect Size

NCSS.com

Power Calculations for One-Way ANOVA
The calculation of the power of a particular test proceeds as follows:
1. Determine the critical value, πΉπΉπΊπΊβˆ’1,π‘π‘βˆ’πΊπΊ,𝛼𝛼 where 𝛼𝛼 is the probability of a type-I error and G and N are defined above. Note that this is a two-tailed test as no direction is assigned in the alternative hypothesis.
2. Choose an effect size parameter and its value.
3. Compute the power as the probability of being greater than πΉπΉπΊπΊβˆ’1,π‘π‘βˆ’πΊπΊ,𝛼𝛼 on a noncentral-F distribution with noncentrality parameter Ξ», where Ξ» is calculated from f or Ξ·2 using the formulas given above.

597-3

Β© NCSS, LLC. All Rights Reserved.

PASS Sample Size Software

One-Way Analysis of Variance F-Tests using Effect Size

NCSS.com

Example 1 – Finding Sample Size
An experiment is being designed to compare the means of four groups using an F test with a significance level of 0.05. The researchers would like to determine the sample size required to detect a small, medium, or large effect when the power is either 0.80 or 0.90.

Setup
If the procedure window is not already open, use the PASS Home window to open it. The parameters for this example are listed below and are stored in the Example 1 settings file. To load these settings to the procedure window, click Open Example Settings File in the Help Center or File menu.

Design Tab
_____________
Solve For .......................................................Sample Size Power.............................................................0.8 0.9 Alpha.............................................................. 0.05 G (Number of Groups) ...................................4 Group Allocation Input Type ..........................Equal (n1 = Β·Β·Β· = nG) Effect Size Type.............................................f = Οƒm/Οƒ f (Οƒm / Οƒ) ........................................................0.1 0.25 0.4

_______________________________________

Output
Click the Calculate button to perform the calculations and generate the following output.

Numeric Reports

Numeric Results

─────────────────────────────────────────────────────────────────────────

Solve For:

Sample Size

Number of Groups: 4

─────────────────────────────────────────────────────────────────────────

Sample Size

─────────

Effect Size

Total Grp ───────────

Power N ni f Ξ·Β² Alpha ───────────────────────────────────────────────────────────────────────

0.80073 1096 274 0.10 0.00990

0.05

0.80399

180

45 0.25 0.05882

0.05

0.82340

76

19 0.40 0.13793

0.05

0.90065 1424 356 0.10 0.00990

0.05

0.90181

232

58 0.25 0.05882

0.05

0.91155

96

24 0.40 0.13793

0.05

─────────────────────────────────────────────────────────────────────────

Power The probability of rejecting a false null hypothesis when the alternative hypothesis is true.

N

The total number of subjects in the study.

ni

The Sample Size per Group is the number of items sampled from each group in the study.

f

The Effect Size. f = Οƒm / Οƒ, where Οƒm is the (sample size weighted) standard deviation of the means and Οƒ is the

standard deviation within a group.

597-4

Β© NCSS, LLC. All Rights Reserved.

PASS Sample Size Software

One-Way Analysis of Variance F-Tests using Effect Size

NCSS.com

Ξ·Β²

The Effect Size. Ξ·Β² = ΟƒmΒ² / (ΟƒmΒ²+σ²), where Οƒm is the (sample size weighted) standard deviation of the means and Οƒ

is the standard deviation within a group. It is the proportion of the total variation (ΟƒmΒ²+σ²) that is explained by the

variation in the means (ΟƒmΒ²). It is analogous to RΒ² in regression.

Alpha The probability of rejecting a true null hypothesis.

Group Sample Size Details

─────────────────────────────────────────────────────────────────────────

n

N

Group Sample Sizes

Group Allocation Proportions

───────────────────────────────────────────────────────────────────────────────────────────

n(1) 1096 274, 274, 274, 274

0.25, 0.25, 0.25, 0.25

n(2)

180

45, 45, 45, 45

0.25, 0.25, 0.25, 0.25

n(3)

76 19, 19, 19, 19

0.25, 0.25, 0.25, 0.25

n(4) 1424 356, 356, 356, 356

0.25, 0.25, 0.25, 0.25

n(5)

232

58, 58, 58, 58

0.25, 0.25, 0.25, 0.25

n(6)

96 24, 24, 24, 24

0.25, 0.25, 0.25, 0.25

─────────────────────────────────────────────────────────────────────────

Summary Statements ───────────────────────────────────────────────────────────────────────── A one-way ANOVA study with a sample of 1096 subjects divided among 4 groups, achieves a power of 80%. This
power assumes a non-central F test with a significance level of 0.05. The group subject counts are 274, 274, 274, 274. The effect size f, which is calculated using f = (Οƒm / Οƒ), is equal to 0.1. Note that the definition of Οƒm includes both the means and the sample sizes. ─────────────────────────────────────────────────────────────────────────

Dropout-Inflated Sample Size

─────────────────────────────────────────────────────────────────────────

Dropout-

Inflated

Expected

Enrollment Number of

Sample Size Sample Size

Dropouts

Dropout Rate

N

N'

D

─────────────────────────────────────────────────────────────────────────────

20%

1096

1370

274

20%

180

225

45

20%

76

95

19

20%

1424

1780

356

20%

232

290

58

20%

96

120

24

─────────────────────────────────────────────────────────────────────────

Dropout Rate The percentage of subjects (or items) that are expected to be lost at random during the course of the study

and for whom no response data will be collected (i.e., will be treated as "missing"). Abbreviated as DR.

N

The evaluable sample size at which power is computed. If N subjects are evaluated out of the N' subjects that

are enrolled in the study, the design will achieve the stated power.

N'

The total number of subjects that should be enrolled in the study in order to obtain N evaluable subjects,

based on the assumed dropout rate. After solving for N, N' is calculated by inflating N using the formula N' =

N / (1 - DR), with N' always rounded up. (See Julious, S.A. (2010) pages 52-53, or Chow, S.C., Shao, J.,

Wang, H., and Lokhnygina, Y. (2018) pages 32-33.)

D

The expected number of dropouts. D = N' - N.

Dropout Summary Statements ───────────────────────────────────────────────────────────────────────── Anticipating a 20% dropout rate, 1370 subjects should be enrolled to obtain a final sample size of 1096 subjects.
─────────────────────────────────────────────────────────────────────────

597-5

Β© NCSS, LLC. All Rights Reserved.

PASS Sample Size Software

One-Way Analysis of Variance F-Tests using Effect Size

NCSS.com

References ───────────────────────────────────────────────────────────────────────── Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates.
Hillsdale, New Jersey Desu, M. M. and Raghavarao, D. 1990. Sample Size Methodology. Academic Press. New York. Fleiss, Joseph L. 1986. The Design and Analysis of Clinical Experiments. John Wiley & Sons. New York. Kirk, Roger E. 1982. Experimental Design: Procedures for the Behavioral Sciences. Brooks/Cole. Pacific Grove,
California. ─────────────────────────────────────────────────────────────────────────
This report shows the numeric results of this power study.
Plots Section
Plots ─────────────────────────────────────────────────────────────────────────

597-6

Β© NCSS, LLC. All Rights Reserved.

PASS Sample Size Software

One-Way Analysis of Variance F-Tests using Effect Size

NCSS.com

These plots give a visual presentation to the results in the Numeric Report.

597-7

Β© NCSS, LLC. All Rights Reserved.

PASS Sample Size Software

One-Way Analysis of Variance F-Tests using Effect Size

NCSS.com

Example 2 – Validation using Fleiss (1986)
Fleiss (1986) page 374 presents an example of determining a sample size in an experiment with 4 groups; means of 9.775, 12, 12, and 14.225; standard deviation of 3; alpha of 0.05, and beta of 0.20. He determines that a sample size of 11 per group is neededβ€”for a total sample size of 44. These means and standard deviation result in a value of 0.5244 for f.

Setup
If the procedure window is not already open, use the PASS Home window to open it. The parameters for this example are listed below and are stored in the Example 2 settings file. To load these settings to the procedure window, click Open Example Settings File in the Help Center or File menu.

Design Tab
_____________
Solve For .......................................................Sample Size Power............................................................. 0.8 Alpha.............................................................. 0.05 G (Number of Groups) ...................................4 Group Allocation Input Type ..........................Equal (n1 = Β·Β·Β· = nG) Effect Size Type.............................................f = Οƒm/Οƒ f (Οƒm / Οƒ) ........................................................0.5244

_______________________________________

Output
Click the Calculate button to perform the calculations and generate the following output.

Numeric Results

─────────────────────────────────────────────────────────────────────────

Solve For:

Sample Size

Number of Groups: 4

─────────────────────────────────────────────────────────────────────────

Sample Size

─────────

Effect Size

Total Grp ─────────────

Power N ni f Ξ·Β² Alpha ──────────────────────────────────────────────────────────────────────────

0.80266

44

11 0.5244 0.21568

0.05

─────────────────────────────────────────────────────────────────────────

Group Sample Size Details

─────────────────────────────────────────────────────────────────────────

n

N

Group Sample Sizes

Group Allocation Proportions

────────────────────────────────────────────────────────────────────────────────────────

n(1) 44 11, 11, 11, 11

0.25, 0.25, 0.25, 0.25

─────────────────────────────────────────────────────────────────────────

PASS also found N = 44. Note that Fleiss used calculations based on a normal approximation, but PASS uses exact calculations based on the non-central F distribution.

597-8

Β© NCSS, LLC. All Rights Reserved.
AnalysisPowerLlcRightsDesign