Reject H₀ at α = 0.05. If all group means were equal, a result at least this extreme would turn up with probability 0.0032. The p-value is not the probability that H₀ is true, and a significant result says nothing about how large or important the effect is. Group membership accounts for 61.5% of the total variation (η²). ANOVA does not say which groups differ — follow up with a post-hoc comparison such as Tukey's HSD.
One-way analysis of variance (ANOVA) tests whether two or more independent groups share the same mean. It splits the total variation into a between-group sum of squares and a within-group sum of squares, divides each by its degrees of freedom (k − 1 and N − k) to get mean squares, and takes their ratio as the F statistic. The p-value is the area of the F distribution beyond that F.
Typical uses are comparing crop yields under three fertilisers, exam scores across teaching methods or response times across product versions. The default data, the NIST e-Handbook example with three groups of five, give F = 9.59 on 2 and 12 degrees of freedom and p = 0.0032, so at α = 0.05 the three means are not all equal.
The test assumes independent observations, roughly normal data in each group and similar group variances. A significant F says that at least one mean differs, not which one.
Checked against: NIST/SEMATECH e-Handbook §7.4.3.3 ANOVA table (SS 27.897 / 17.452, F = 9.59); p = (12/(12 + 2F))⁶ by A&S 26.6.4 in Python
Unequal group sizes
Groups
23 25 21 22
28 30 27 26 29
24 26 25
Significance level α
0.05
F statistic
13.5
df between groups
2
df within groups
9
p-value
0.001953
η² (share of variance explained)
0.75
Checked against: Sums of squares with Python fractions (SSB 62.25, SSW 20.75); A&S 26.6.4 for d₁ = 2 gives p = (9/(9 + 2F))^4.5 = 4^−4.5 = 1/512
Two groups equals the pooled t-test
Groups
12 15 11 14 13 16
17 14 18 16 19 15
Significance level α
0.05
F statistic
7.714286
df within groups
10
p-value
0.019536
Checked against: F = t² for two groups; pooled t from Python fractions, two-sided p from the A&S 26.7.4 closed form with ν = 10
Edge case: identical group means
Groups
1 2 3
2 1 3
3 2 1
Significance level α
0.05
F statistic
0
p-value
1
Decision
Fail to reject H₀
Checked against: Every group mean is 2, so SS between = 0 and F = 0
Questions
What does the p-value in ANOVA mean?
It is the probability of an F statistic at least as large as the one observed if every group mean were equal. With the default data p = 0.0032: equal means would produce F ≥ 9.59 in about 3 samples out of 1,000. It is not the probability that the null hypothesis is true, and a small p-value does not say how large the differences are; η² measures that.
How do you interpret the F statistic?
F is the between-group mean square divided by the within-group mean square. Equal population means give F values near 1; larger values point to real differences. How large is large enough depends on the degrees of freedom: with 2 and 12 df the 5% critical value is 3.885, so F = 9.59 is significant at α = 0.05 and F = 3 would not be.
What are the assumptions of one-way ANOVA?
Independent observations, a roughly normal distribution in each group, and equal population variances. The F test tolerates moderate non-normality when groups are of similar size. Moore and McCabe's rule of thumb accepts the equal-variance assumption if the largest group standard deviation is less than twice the smallest; otherwise use Welch's ANOVA, and for clearly non-normal data the Kruskal–Wallis test.
How do you find which groups differ after ANOVA?
Run a post-hoc test. Tukey's honestly significant difference (HSD) compares every pair while holding the family-wise error rate at α; the Bonferroni method tests each of the m pairs at α/m, which is 0.05/3 ≈ 0.0167 for three groups. Separate t-tests at 0.05 on every pair push the chance of at least one false positive well above 5%, towards 1 − 0.95³ ≈ 14% for three comparisons.
What is a good eta squared value?
η² is the between-group sum of squares divided by the total sum of squares: the share of variation explained by group membership. Cohen (1988) proposed 0.01, 0.06 and 0.14 as small, medium and large effects. The default data give 27.897/45.349 = 0.615, a very large effect. η² overstates the population effect in small samples; ω² corrects for that bias.
How accurate is the one-way ANOVA calculator?
Accuracy depends on your inputs and the method's assumptions. Decimal arithmetic uses 50 significant digits, but estimates, numerical methods and source data can be less precise; the displayed rounding does not remove those limits. It is checked against 4 worked examples whose answers come from independent sources; for example, “NIST e-Handbook example, 3 groups of 5 (defaults)” is checked against NIST/SEMATECH e-Handbook §7.4.3.3 ANOVA table (SS 27.897 / 17.452, F = 9.59); p = (12/(12 + 2F))⁶ by A&S 26.6.4 in Python.
Where does the method come from?
NIST/SEMATECH e-Handbook of Statistical Methods, §7.4.3 Are the means equal? (one-way ANOVA and worked example); Abramowitz & Stegun, Handbook of Mathematical Functions, §26.6 (F distribution).