ANOVA Calculator - One-Way F-Test, Welch and Effect Size

Problem: three group means, and three t-tests that overstate the evidence

Three fertilizers, three average corn yields: 549.4, 557.5 and 722.4. The tempting move is to run a t-test on every pair. With three groups that is three tests, and with every extra group the number of pairs grows fast: 6 pairs for four groups, 10 for five, 45 for ten. Each test carries its own 5% chance of a false alarm, so the chance that at least one of them fires by accident is far above 5%. A one-way ANOVA answers the first question with a single test: is there any difference among the means at all? This calculator runs that test from raw measurements or from a table of sizes, means and standard deviations, and adds the two checks a plain F-test leaves out.

The second trap. Classic ANOVA assumes the groups have equal variances. When one group is far more spread out than the others, the F-test can call a real difference noise. In the example further down, the classic F gives p = 0.6443 and the Welch version of the same test gives p = 0.0197.

Solution: one F-test, a Welch check and an equal-variance test

Type of data - raw measurements when you have every value, summary statistics when a paper or a report gives only the size, mean and standard deviation of each group.
Measurements of each group - one group per line, values separated by spaces, commas, semicolons or tabs, a dot for decimals. An optional name goes before a colon: Fertilizer A: 452 874 554. From 2 to 10 groups, up to 2,000 values each.
Group statistics - in the summary mode, one line per group with three numbers: size, mean, sample standard deviation (the one with n - 1, which is what STDEV.S in Excel and sd() in R return), for example 13, 549.38, 168.76.
Significance level - 0.05 in most fields, 0.01 when a false alarm is costly, 0.10 for a first look. Pick it before you see the result.
Read the result - F with its degrees of freedom and the p-value, the full ANOVA table, η² and ω², the critical F, the Welch test, the Brown-Forsythe test of equal variances (raw data only) and a warning when the variances look unequal.

Which of the three results goes in your report

The calculator shows all three tests at once, so the choice is about which line goes into your report. The rules below follow the NCSS documentation for one-way ANOVA, which also quotes the rule of thumb used here: some authors switch to Welch when the largest group variance is more than four times the smallest.

What you see	Report	Why
Brown-Forsythe p ≥ 0.05, variance ratio under 4	Classic F	No sign that the equal-variance assumption fails
Brown-Forsythe p < 0.05, or ratio above 4	Welch F	Welch does not pool the variances, so one noisy group cannot hide a difference
Only summary statistics	Classic F, or Welch if the SDs differ a lot	Without raw values no variance test is possible; the ratio rule still works
Clearly skewed data, ranks or ratings, equal spreads	Kruskal-Wallis	A rank test does not need normal data
Significant result with three or more groups	A post hoc test next	ANOVA says some means differ, not which ones

NCSS adds a caution worth repeating: the Kruskal-Wallis test also assumes equal variances, so it is not the cure for unequal spreads. With normal data and unequal variances, Welch is the tool.

Five data sets, run through the calculator

The textbook example: 6 8 4 5 3 4 / 8 12 9 11 6 8 / 13 9 11 8 7 12 - group means 5, 9 and 10, SS between 84, SS within 68, F(2, 15) = 9.2647, p = 0.0024. The critical F at 0.05 is 3.6823, and the groups explain 55.3% of the variation (η² = 0.553).

Corn yield for three fertilizers (13, 16 and 14 plots) - F(2, 40) = 7.474, p = 0.0017; Welch F = 8.0236 with 24.27 denominator degrees of freedom, p = 0.0021; Brown-Forsythe W = 1.0866, p = 0.3471, so the classic result stands. These match the NCSS worked example to the last printed digit.

One noisy group: 10 11 9 10 10 / 2 30 15 1 25 / 12 13 11 14 12 - the classic F is 0.4561, p = 0.6443. Brown-Forsythe p = 0.0011 flags the unequal spreads, and Welch gives F = 7.478, p = 0.0197: once the noisy second group stops inflating the pooled variance, the difference among the means shows.

Four groups from a summary table: 25, 72.4, 8.1 / 25, 70.1, 7.6 / 25, 76.9, 9.3 / 25, 71.8, 8.8 - F(3, 96) = 2.9307, p = 0.0375, while Welch gives p = 0.0559. The variance ratio is only about 1.5, so there is no warning, but a result this close to the line is a reason to report both.

Two groups: 1 2 3 / 4 5 6 - F(1, 4) = 13.5, p = 0.0213. With two groups ANOVA is the pooled t-test in disguise: t = 3.674, and t² = 13.5.

Critical F at 0.05 and 0.01 for 2 to 6 groups

The numerator degrees of freedom are the number of groups minus one; the denominator degrees of freedom are the total number of observations minus the number of groups. Your F is significant when it is larger than the value in the table. The 0.05 rows agree with the NIST/SEMATECH e-Handbook F table to all three decimals.

Denominator df	2 groups	3 groups	4 groups	5 groups	6 groups
10, α = 0.05	4.965	4.103	3.708	3.478	3.326
20, α = 0.05	4.351	3.493	3.098	2.866	2.711
30, α = 0.05	4.171	3.316	2.922	2.690	2.534
60, α = 0.05	4.001	3.150	2.758	2.525	2.368
100, α = 0.05	3.936	3.087	2.696	2.463	2.305
20, α = 0.01	8.096	5.849	4.938	4.431	4.103
60, α = 0.01	7.077	4.977	4.126	3.649	3.339

How the F statistic is put together

The total variation of all values around the grand mean splits into two parts. SS between adds up, for every group, its size times the squared distance of its mean from the grand mean. SS within adds up the squared distances of each value from its own group mean. Dividing each by its degrees of freedom gives the mean squares, and F = MS between / MS within. If the groups come from populations with the same mean, F hovers around 1; a large F means the group means are further apart than the scatter inside the groups would explain. η² is SS between over SS total; ω² = (SS between - df between × MS within) / (SS total + MS within) takes out the part of SS between that chance alone would produce, so it is usually a little smaller.

Welch's version weights each group by n / s² instead of pooling the variances, and corrects the denominator degrees of freedom, which is why the corn example shows 24.27 rather than 40. Brown-Forsythe runs an ordinary ANOVA on the absolute distances of each value from its group median; a small p there means the spreads differ.

ANOVA questions from coursework and lab reports

How do I write the result in APA style?

Give F with both degrees of freedom, the p-value and an effect size: for the textbook example, F(2, 15) = 9.26, p = .002, η² = .55. For the Welch test the denominator degrees of freedom are usually not whole numbers: Welch's F(2, 24.27) = 8.02, p = .002.

Can I run ANOVA from means and standard deviations only?

Yes. The F statistic needs only the size, mean and standard deviation of each group, so the summary mode gives the same F as the raw data: the NCSS corn summary 13, 549.3846, 168.7629 / 16, 557.5, 104.6219 / 14, 722.3571, 127.8873 returns F = 7.47. What you lose is the Brown-Forsythe test and the medians, because both need the individual values.

Do the groups have to be the same size?

No. The corn example has 13, 16 and 14 plots. Unequal sizes matter more when the variances also differ, because the classic F then leans on the larger groups; that is the situation where the Welch result deserves the most weight.

What is a good η²?

It depends on the field more than on any universal scale. Read it literally: η² = 0.272 in the corn data means the fertilizer accounts for 27.2% of the variation in yield and the remaining 72.8% sits inside the groups. ω² is shown next to it because it subtracts what chance alone adds to SS between: 0.231 for the same data.

Does this handle repeated measures or two factors?

No. This is one-way ANOVA for independent groups: every value belongs to one group and no person or plot appears twice. The same people measured at several time points need a repeated-measures design, and two crossed factors need a two-way ANOVA.

Related tools

Kruskal-Wallis Test Calculator

The rank-based comparison of several groups, with an exact p and Dunn pairs - See the calculator

Tukey HSD Calculator

Which pairs of group means differ after the ANOVA, with simultaneous intervals - See the calculator

Two-Sample T-Test Calculator

Two groups only, with Welch, Student and paired versions side by side - See the calculator

P-Value Calculator

Turns an F printed in a paper, with its two degrees of freedom, into a p-value - See the calculator

Wilcoxon Test Calculator

The rank-based test for two groups when the data are far from normal - See the calculator

Sample Size Calculator

How many observations a planned comparison of means needs before you collect them - See the calculator

Confidence Interval Calculator

An interval around each group mean, for the chart that goes next to the ANOVA table - See the calculator

Chi-Square Calculator

The counterpart of ANOVA when the outcome is a category rather than a measurement - See the calculator