> Here

The Rationale of Inferential Statistics

The intent of the page

I created this page as a way to introduce the "backward" way of thinking adopted by the Hypothesis Testing Model of Inferential Statistics. I have avoided most technical terminology, choosing to focus on the concepts of the null and alternative hypotheses.

The Phenomenon of Interest

Simple Visual Reaction Time (SVRT) is the length of time it takes to respond to a visual stimulus. The simplest situation: (1) the participant's finger is poised over a button; (2) the subject is looking at a small light; (3) the subject is instructed to press the button as soon as the light comes on; (4) the reaction time is the length of time between when the light came on and the button was pressed; (5) the SVRT is the average reaction time over several hundred trials.

The Research Question

Do males and females have the same Simple Visual Reaction Time?

The "Experiment"

Randomly select 4 college aged males and females. Measure the reaction time on each subject.

The Results

The Female particpants had an average SVRT of 0.179 seconds. The males had an average SVRT of 0.190 seconds. Typically simple reaction times are expressed in milliseconds (ms), so the results are an average female reaction time of 190 ms and an average male reaction time of 179 ms; that is, the average female reaction time was 11 ms longer than the average male reaction time1. These results are depicted here:

Difference in Average SVRT
4 females and 4 males.

For these 8 subjects, the 11 ms difference in SVRT is a fact--the 11 ms difference is a descriptive statistic.

Here is the pertinent question: Is there a difference in the SVRT of males and females in general.?

The results we got could have arisen from two different situations.

The "Null Hypothesis" (Ho): On the average, males and females have the same reaction time. The true difference in SVRT for females and males is 0 ms (also known as a "null" difference). The 11 ms difference in this group of subjects is simply random error or sampling error.

The "Alternativve Hypothesis" (Ha): On the average, males and females have different reaction times. The true difference in SVRT for females and males in not 0 ms. There is a difference in reaction times and the difference measured in this group of subjects

How can we decide which hypothesis (Ho vs Ha) corectly describes the difference in female and male Simple Visual Reaction Time? There are three possibilities:

1. Repeat the research with many randomly selected groups of subjects. Plot each result and interepret the results of many many replications.

2. Hire a consultant to look at the research results and tell you her opinion. The problem here is we don't have any idea of how often the consultant's opion is right or wrong. Of the last 100 jobs, how many times did she give advice that was later proven to be false?

3. Use inferential statistics.

Question: So, Ms Inferential Statistics, is there a difference in SVRT for males and females?

Answer: What would the results look like if there were no difference in RT for the sexes?

Question: OK, so.....?

Answer: Let's set up simulated experiments and repeat the experiment many times and see what the results look like. We can use the standard deviation from the results of your experiment to get an idea of scattered the scores are and arbitrarily set the sex difference to 0 ms.

Question: And...."

Answer: Here's the results we get for 100 replications of your experiment when we know the null hypothesis is true.Click the controller to watch as the research is replicated.

So, What do you think? Is the 11 ms difference true of people in general (Ha) or does it appear that there really isn't a difference in RT between the sexes?

As you look at the completed replications you might decide that the 11 ms difference you actually truly measured could have come from the null-hypothesis situation. The data are not clearly, unambiguously showing a sex difference in RT. There might be a difference. There might not be a difference.

Now, change the situation slightly. Use a larger sample. Pretend your results (females: 190 ms; maes: 179 ms) was based on 9 or 25 subjects in each group Now complete 100 simulated replications for the situation where the null hypothesis is true.

100 Simulated Replications with Varying Sample Size
n = 4 per group
n = 9 per group
n = 25 per group

The way I would interpret these results is as follows:

If I collected data on 4 subjects per group, I would conclude there is no evidence the "no difference" hypothesis is false. The data are the sort of results that could happen under "no difference.

If I collected data on 25 subjects per group, I would conclue there is evidence the "no difference" hypothesis is false. There really is a difference between female and male reaction times.

If I collected data using 9 subjects per group, I'd still have a rough time making a decision. That's why the decision making process has been refined whenever I use inferential statistics to decide whether or not a research result would be repeated with new groups of subjects. The refinements involve Type-I errors, Type-II errors, and decision rules. Other tutorials deal with these

1. There is a difference in SVRT in males and females. It is an easily replicated phenomenon. The difference is something like 15 or 20 ms with females having a longer reaction time.

Here is a way you can replicate the result with a minimum of equipment. All you need is a yard or meter stick and a smooth wall.

©2006 by Burrton Woodruff. All rights reserved. Modified 3/22/07