Formal Hypothesis Testing

Hypothesis testing is a formal procedure often employed in scientific research to test theories or models. In effect the researcher bets in advance of his experiment that the results will agree with his theory and cannot be accounted for by the chance variations that are involved in the sampling.

The following is a formal procedure for the statistical testing of theories:

1. Plan an experiment so that if the results cannot be explained by chance variation involved in drawing the sample, then your theory will be confirmed.
2. Conduct the experiment and collect the data.
3. Assume that the results are due to chance alone. This is called the NULL HYPOTHESIS.
4. Use a theoretical sampling distribution based on the null hypothesis to determine the probability of obtaining sample data like yours by chance alone.
5. If the probability of obtaining sample data like yours by chance is less than some predetermined small percentage (usually 5% or 1%) the results will be significant. You may reject the null hypothesis and accept your alternative hypothesis (your theory).

The logic of the hypothesis, in sum, sets up two opposing hypotheses, the null hypothesis and alternate. The null is established as the opposite of what the investigator really wants to conclude. If we reject the null then we conclude that there is a difference.

Formal hypotheses, also called statistical hypotheses, refer to populations that represent ways by which findings are generalized from the sample to the population. Here the researcher's challenge is to determine if the means of the samples are sufficiently different to support the decision that a true difference exits between the two populations. The researcher assumes that the two samples come from the same population and have identical means. Therefore, any observed different between the two samples must have occurred by chance. These assumptions are called the null hypothesis.

To prove that the two means are significantly different, and not due to chance, the researcher will determine a level of significance. The level of significance communicates the probability that the researcher erred in rejecting the null hypothesis. Common levels of significance used are 0.05, 0.01 and 0.001. At the 0.05 level of significance, the researcher is saying that the probability of error in rejecting the null hypothesis is 5/100. Researchers will then use a statistical tool, such as a t-test, to show if the data they collected occurred greater than 5/100 and if it does the null hypothesis would be rejected and implies no significant difference between the two means; on the other hand if 5/100 or less, this would shows significant difference between the means . The null hypothesis would be rejected and the results would be significant.

Since these are statistical hypotheses we could make an error in our inference (unbelievable!). Later in the statistics portion of this unit we will learn that, regarding the null hypothesis, there are two kinds of errors we can make. The first kind of error - concluding that the null hypothesis is false when it is actually true - is a Type I error. The converse kind of error (accepting the null when it is really false) is called (what else!) a Type II error. To summarize:

• Null hypothesis - assumption that the experimental results are due to chance.
• Alternative hypothesis - your theory that will be confirmed if you reject the null hypothesis.
• Significant results - experimental results that are not likely to have occurred by chance alone.