Power and statistics
One of the common questions is - which test is best for an experiment? Well, this depends on many different factors.
One of the key ideas of statistics is that tests have different "power" - that is, the probability of making a correct decision (to reject the null hypothesis) when the null hypothesis is false. Power is the probability that a test of significance will pick up on an effect that is present.
If you have interval or ratio data, you could carry out a range of tests, each with different levels of power. So, why would you make the decision to use a lower-power test?
- How many levels of the independent variable are there? The IB recommends for the IA that students only use two levels of the IV. If there are more, more sophisticated tests must be applied.
- How big is your sample? The recommendation for a parametric test (e.g. t-test) is 30 or more. Most students' IA samples are lower than 30, so non-parametric tests (e.g. Mann Whitney) are more appropriate.
- Is your sample normally distributed? This means that the empirical distribution of the data (a histogram) should be bell-shaped and resemble the normal distribution. If using a small sample, however, this strategy is often not enough and tests to determine normal distribution must be used. For the IA, it is often easier to just use a non-parametric test (e.g. Mann Whitney).
- Which design was used? There are different tests depending on whether the design was independent samples (e.g. chi-squared, Mann Whitney, independent t-test) or repeated measures (sign test, Wilcoxon signed ranks, dependent t-test).
Presentation
The following presentation walks you through an experiment commonly done for the internal assessment - Loftus and Palmer (1974). You will see that you could carry out an ANOVA, t-test, Mann Whitney or Chi-Squared for the experiment, depending on several of the factors listed above.
Helpful links
The following sites can be used to carry out inferential statistics.