t-test calculator for one-sample, two-sample (Welch or pooled) and paired tests, plus z-tests for a mean or proportion: p-value, critical value, decision.
By AdminDrafted with AI writing toolsUpdated Checked examples: 7
Reject H₀ at α = 0.05. If the true mean were 500, a result at least this extreme would turn up with probability 0.0355. The p-value is not the probability that H₀ is true, and a significant result says nothing about how large or important the effect is.
Reject H₀
For a two-sided test, compare |t| with 2.201.
A t-test divides the gap between a sample mean and a hypothesised value, or between two sample means, by its standard error. The ratio follows Student's t distribution with n − 1 degrees of freedom for one sample or for paired differences, n₁ + n₂ − 2 for the pooled two-sample test, and the Welch–Satterthwaite value when the variances may differ. With a known population σ the same ratio is a z statistic; the proportion test compares a sample proportion with p₀ the same way.
Quality checks, A/B tests and before-and-after studies are typical uses. The default data are 12 fill weights tested against a target of 500: the mean is 498.64, t = −2.395 on 11 df and the two-sided p = 0.0355, so the mean differs from 500 at α = 0.05.
For independent-sample t-tests, observations must be independent and the populations approximately normal; the pooled test also assumes equal variances. For paired tests, pairs are independent and their differences should be approximately normal. With finite variance and suitable sampling, larger samples can tolerate moderate non-normality.
A one-sample t-test computes t = (x̄ − μ₀) ÷ (s ÷ √n), the distance from the sample mean to the hypothesized mean in standard errors, on n − 1 degrees of freedom. Twelve fill weights averaging 498.64, tested against 500, give t = −2.395 and a two-sided p-value of 0.0355.
The fill weights are 498.2, 501.3, 497.6, 499.1, 495.8, 500.4, 496.9, 498.7, 497.3, 499.8, 502.1 and 496.5. Worked by hand:
At α = 0.05 the two-sided critical value is 2.201. Since |t| = 2.395 exceeds it, and equivalently p ≤ 0.05, the test rejects H₀: the mean fill differs from 500.
For two independent groups, the statistic compares the difference in means with its standard error. The two versions differ in how they get that standard error:
Use Welch unless there is good reason to believe the population variances are equal. It loses little power when they are, while the pooled test's false-positive rate drifts away from α when both the variances and the group sizes differ (NIST).
Take means 20.1 and 17.4, SDs 3.2 and 4.8, and group sizes 15 and 12:
Neither result is significant at 0.05. Here the SDs and the group sizes both differ, the case Welch is designed for, and a smaller p-value is no reason to switch to pooled.
When each value in one sample is matched to one in the other, such as the same people measured before and after, test the differences. Subtract within each pair, then run a one-sample t-test of the differences against Δ₀, usually 0.
Take eight pairs: 142 vs 136, 138 vs 135, 150 vs 146, 145 vs 144, 160 vs 150, 155 vs 149, 139 vs 138 and 148 vs 141. The differences are 6, 3, 4, 1, 10, 6, 1 and 7.
Their mean is 4.75 with SD 3.105, so t = 4.75 ÷ (3.105 ÷ √8) = 4.33 on 7 df, and the right-sided p-value is 0.0017.
Run as two independent samples, the same numbers give a right-sided Welch p-value of 0.0935. Pairing removes the person-to-person spread: the first measurements have an SD of 7.72, the differences only 3.11.
With a population σ known from outside the sample, the statistic is z = (x̄ − μ₀) ÷ (σ ÷ √n), read from the standard normal.
A mean of 103 from 36 values with σ = 15, tested against 100, gives z = 3 ÷ 2.5 = 1.2. The right-sided p-value is 0.1151, short of significance at 0.05, where z must exceed 1.645.
A one-proportion z-test uses the null proportion in the standard error: z = (p̂ − p₀) ÷ √(p₀(1 − p₀) ÷ n). For 540 successes in 1,000 against p₀ = 0.5, z = 0.04 ÷ 0.015811 = 2.530, with a two-sided p-value of 0.0114.
The proportion test is a normal approximation without a continuity correction. When n·p₀ or n·(1 − p₀) is below 10, an exact binomial test is safer.
Use a two-sided test unless the direction was fixed before seeing the data and an effect the other way would count the same as no effect. For the fill weights, the left-sided p-value is 0.0178, half the two-sided 0.0355. The right-sided p-value is 0.9822.
Choosing the tail after looking at the data doubles the real false-positive rate. Every version rejects when p ≤ α.
The p-value is the probability of a statistic at least as extreme as the one observed, if H₀ is true. For the fill weights, samples at least that far from 500 would turn up about 3.6% of the time if the true mean were 500.
It is not the probability that H₀ is true, and it does not measure the size of the effect. Failing to reject H₀ does not show the means are equal; the sample may be too small to detect a real difference.
Observations must be independent, and each population should be roughly normal; for a paired test, the differences should be. The pooled test also assumes equal variances.
Larger samples tolerate moderate non-normality, but strong skew or outliers in a small sample can still mislead.
| Scenario | Alternative | p-value |
|---|---|---|
| Default fill weights versus 500 | Two-sided | 0.035527 |
| Independent summaries, Welch | Two-sided | 0.111135 |
| Same summaries, pooled variance | Two-sided | 0.092410 |
| 540/1000 successes versus 0.5 | Two-sided | 0.011412 |
| Known σ = 15, mean 103, n = 36, null 100 | Right-sided | 0.115070 |
Each row is worked in a section above. The independent summaries are means 20.1 and 17.4, SDs 3.2 and 4.8, and sizes 15 and 12.
For an interval estimate of the same difference, use the confidence interval calculator. Three or more groups need the one-way ANOVA calculator, and planning n for a given power is done in the sample size calculator.
Checked against: p-value from the A&S 26.7.3 closed form for odd ν; t₀.₉₇₅,₁₁ = 2.201 (t table)
Checked against: Welch–Satterthwaite df in with exact fractions; p-value by Gauss–Legendre quadrature of the t density
Checked against: Pooled variance in with exact fractions; p-value from the A&S 26.7.3 closed form with ν = 25
Checked against: p-value from the A&S closed form with ν = 7
It is the probability of a test statistic at least as extreme as the one observed, assuming the null hypothesis is true. For the default data p = 0.0355: if the true mean were 500, samples at least this far from 500 would turn up about 3.6% of the time. It is not the probability that H₀ is true and it does not measure effect size, as the American Statistical Association's 2016 statement on p-values stresses.
Use Welch's test unless you have good reason to believe the variances are equal. It drops the equal-variance assumption and loses little power when the variances do match, while the pooled test's false-positive rate drifts from α when variances and group sizes both differ. On the worked example (s = 3.2 and 4.8, n = 15 and 12) Welch gives p = 0.111 on 18.4 df and the pooled test p = 0.092 on 25 df.
When each value in one sample is matched to one in the other: the same patients before and after treatment, or two instruments measuring the same parts. The test is a one-sample t-test on the differences, which removes the variation between subjects. In the paired worked example eight pairs differ by 4.75 on average, giving t = 4.33 on 7 df and a one-sided p of 0.0017.
A z-test uses a known population standard deviation σ and the standard normal distribution; a t-test estimates σ from the sample and uses Student's t, whose heavier tails allow for that extra uncertainty. The two-sided 5% critical value is 1.960 for z, 2.201 for t with 11 df and 2.042 with 30 df. σ is rarely known in practice, so the t-test is the usual choice for means.
Use a two-tailed test unless the direction was fixed before seeing the data and an effect in the other direction would be treated the same as no effect. A one-tailed test puts all of α in one tail, so its p-value is half the two-tailed one when the effect goes the predicted way: t = −2.395 on 11 df gives 0.0355 two-tailed and 0.0178 left-tailed. Picking the tail after looking doubles the real false-positive rate.
This calculator includes 7 worked examples with independently sourced answers. They run in the test suite, and you can run them here too.