Home

Contact me

About me

Services

Feedback

Events

Here is a small part of the section about “Tests of significance” from “Fug’s Guide to Statistics”, I hope you like it!!

Tests of significance

It is worthwhile spending time on this topic, as you will need to know this concept really well from the beginning.  There are lots of places to get the wrong idea and get totally lost.  We have found that we can estimate the population mean, standard deviation or any other values of a population from a sample. Next we found that we can find the probability of obtaining certain values or average values from a population for which the mean and standard deviation are known.  Now it is appropriate to use a sample to tell us something about the population and that is what tests of significance do.  In this section, we will explain the logic by using the average value, as it’s the easiest one to understand.

We will take an example and explain each step as we go

“ A researcher thinks that the mean score on a standard test for hand strength should be raised from 33.5 standard units as she believes that children are stronger now than in previous years.  A sample was taken from a local school and the mean hand strength was found to be 35 standard units”

How do we test this claim?

It is not practical to test the hand strength of every child, as it can be time-consuming and frustrating for both the examiner and the child.  The first step is to take a sample and pretend it comes from the only population that we know anything about and for our example we will know the Population Mean. Now this population is usually the one that you hope it has changed from.  That may sound strange but up to now you had believed that value and it may only be recently that suspicions have been raised about the value.  The next step is to see what the probability is of our sample really coming from that particular population.

We will then check the evidence and make a decision.  This is the outline of a test of significance.

A step-by-step guide to the logic behind the test

Step One

A claim is made changing what has been considered to be true in the past.

In our example, the researcher thinks that the mean hand strength is higher than before i.e. that the children are stronger.

Step Two

A random sample is chosen and the appropriate sample statistic is found.  We have some assumptions that must be met before you start the tests and these change according to the tests that are being performed.  We will talk about them later. 

In our example, the sample statistic that will be needed is the sample mean, `x and possibly the sample standard deviation, s, if the population standard deviation, s  , is not known.  In other tests there will be other sample statistics such as proportions, variances and differences of means etc.

Step Three

Now we have to know the sampling distribution of the particular sample statistic that we chose in Step Two.  This could be the Normal distribution, Student’s t distribution or any other distribution.  Although this sounds really difficult, after a while you will not give it a second thought.  The Central Limit Theorem helps us by saying that as long as the sample is large enough then the sampling distribution will be normally distributed no matter what the distribution of the parent population is. We usually check to see if the sample is normally distributed as it is meant to reflect the distribution of the parent population.  Why do we need the sampling distribution of the statistic?  We have to work out the probability of finding a sample statistic of that value or more extreme, from a population where the parameter is that of past years.  Ok, what does that mean in plain language?  We are going to check how likely the value we found in our sample comes from a population we have trusted in the past.

Well in our example, the sample mean is 35 standard units, this is a real value obtained from an experiment.  The population mean is thought to be 33.5 standard units, this is a value that has been true in the past but we are wondering if it is still true.  So let us see what is the probability that the mean hand strength or more comes from a population with mean hand strength of 33.5 standard units.  Why do we need the more bit? It is not possible to find a probability of an exact value of continuous distribution such as the Normal distribution, we are able to calculate the probability of less than or greater than that value.  We don’t need to know any more than this at the moment.

Step Four

If we find that the probability is large, then the sample value or more extreme values are values that would occur in many samples taken from a population. The population is the one known from the past.  If this is the case then the evidence suggests that the population value has not changed from the one in the past.  Remember that when you take samples from a population, the sample values can be smaller or larger than the population value but they all come from the same population.  We would hope that the sample value is close to the population value and good experimental design helps this.

You will possibly want and need to read this bit several times to try to understand this concept and I will give lots of different explanations from time to time.

If, however, the probability is small, then our sample or more extreme values are values that do not occur very often in samples that are taken from the population in question.  The evidence suggests that the population value has changed from that in the past.

In our example, we want to know whether the probability is large or small of getting a sample mean of 35 standard units or greater in a population where the population mean is 33.5 standard units.

Now if the probability is large, it indicates that the sample mean (35) is close enough to the proposed population mean (33.5) for us to believe that the population mean could still be 33.5.

Now if the probability is small, it indicates that the sample mean of 35 is far enough away from the population mean of 33.5 for us to feel that it may come from different population.

This is the spot where you are desperate to say that the population mean is 35 rather than 33.5.  This is not correct as the value of the population mean was gathered from past knowledge, and the sample mean gives evidence for or against this population mean.  We are not interested in quoting about samples but we look to make a statement about the population.  Remember sample values vary from sample to sample whereas the population values are constant for that particular population.

If you want to order this or any other section, contact Anne