4.11.1 Hypothesis Testing

What is a hypothesis test?

• A hypothesis test uses a sample of data in an experiment to test a statement made about the population
  • The statement is either about a population parameter or the distribution of the population
• The hypothesis test will look at the probability of observed outcomes happening under set conditions
• The probability found will be compared against a given significance level to determine whether there is evidence to support the statement being made

What are the key terms used in statistical hypothesis testing?

• Every hypothesis test must begin with a clear null hypothesis (what we believe to already be true) and alternative hypothesis (how we believe the data pattern or probability distribution might have changed)
• A hypothesis is an assumption that is made about a particular population parameter or the distribution of the population
  • A population parameter is a numerical characteristic which helps define a population
    • Such as the mean value of the population
  • The null hypothesis is denoted and sets out the assumed population parameter or distribution given that no change has happened
  • The alternative hypothesis is denoted and sets out how we think the population parameter or distribution could have changed
  • When a hypothesis test is carried out, the null hypothesis is assumed to be true and this assumption will either be accepted or rejected
    • When a null hypothesis is accepted or rejected a statistical inference is made
• A hypothesis test will always be carried out at an appropriate significance level
  • The significance level sets the smallest probability that an event could have occurred by chance
    • Any probability smaller than the significance level would suggest that the event is unlikely to have happened by chance
  • The significance level must be set before the hypothesis test is carried out
  • The significance level will usually be 1%, 5% or 10%, however it may vary

What are one-tailed tests?

• A one-tailed test is used for testing:
• Whether a distribution can be used to model the population
• Whether the population parameter has increased
• Whether the population parameter has decreased
• One-tailed tests can be used with:
• Chi-squared test for independence
• Chi-squared goodness of fit test
• Test for proportion of a binomial distribution
• Test for population mean of a Poisson distribution
• Test for population mean of a normal distribution
• Test to compare population means of two distributions

What are two-tailed tests?

• A two-tailed test is used for testing:
• Whether the population parameter has changed
• Two-tailed tests can be used with:
• Test for population mean of a normal distribution
• Test to compare population means of two distributions

How do I decide whether to reject or accept the null hypothesis?

• A sample of the population is taken and the test statistic is calculated using the observations from the sample
  • Your GDC can calculate the test statistic for you (if required)
• To decide whether or not to reject the null hypothesis you first need either the p-value or the critical region
• The p - value is the probability of a value being at least as extreme as the test statistic, assuming that the null hypothesis is true
  • Your GDC will give you the p-value (if required)
  • If the p-value is less than the significance level then the null hypothesis would be rejected
• The critical region is the range of values of the test statistic which will lead to the null hypothesis being rejected
  • If the test statistic falls within the critical region then the null hypothesis would be rejected
• The critical value is the boundary of the critical region
  • It is the least extreme value that would lead to the rejection of the null hypothesis
  • The critical value is determined by the significance level

How should a conclusion be written for a hypothesis test?

• Your conclusion must be written in the context of the question
• Use the wording in the question to help you write your conclusion
  • If rejecting the null hypothesis your conclusion should state that there is sufficient evidence to suggest the null hypothesis is unlikely true
  • If accepting the null hypothesis your conclusion should state that there is not enough evidence to suggest null hypothesis is unlikely true
• Your conclusion must not be definitive
  • There is a chance that the test has led to an incorrect conclusion
  • The outcome is dependent on the sample
    • a different sample might lead to a different outcome
• The conclusion of a two-tailed test can state if there is evidence of a change
  • You should not state whether this change is an increase or decrease
  • If you are testing the difference between the means of two populations then you can only conclude that the means are not equal
    • You can not say which population mean is bigger
    • You’d need to use a one-tailed test for this