Interpreting Statistical Data
Statistical analyses seem complicated, but they are really pretty simple.
We are measuring a few individuals in each treatment group and trying
to make statements about all of the individuals in that group – without
measuring them all. We gather data on a few and make “inferences”
or estimates about a larger group that we did not measure. This is essentially
the same as measuring the height of all the children in four third-grade
classes in a city, and then based on these four classes, trying to describe
the “average” height of all third graders in the city.
While we may be quite proficient at conducting research, we are not always the best at explaining what we have discovered. Here are a few pointers to help decipher results when we have not explained them as well as we might have.
A) How to interpret whether treatments are really different.
Differences in treatments are often shown in tables with the average for each treatment immediately followed by a letter. For example, the following table presents averages and letters that tell whether a particular treatment was “significantly different” from other treatments. Here we will assume that a higher average means a better product.
| TREATMENT |
AVERAGE |
STD. DEVIATION |
| Product U |
10.5 a |
2.1 |
| Product V |
8.1 a |
1.3 |
| Product W |
7.4 ab |
2.9 |
| Product X |
5.0 b |
1.6 |
| Product Y |
2.0 c |
1.2 |
| Untreated check |
2.8 c |
1.1 |
|
|
|
In this case, we would conclude the following about these different treatments.
1. Product U, V, and W are not statistically different from each other because they are all followed by the letter “a”. This means that while product U has the highest numerical average performance, it is not really significantly different from either product V or W. This lack of statistical difference is based on the variability of the numbers that went in to making the “average” response for each treatment reported in the table. Averages can be deceiving because we do not know how different the numbers were that went in to making the “average”. These differences are describes as “variance”, usually represented by a term called “standard deviation”, which is represented as a + or – (plus or minus).
2. Product W is not statistically different from product U, V, or X. Product W is not different from products U and V because it is followed by the letter “a”. Additionally, it is not different from product X because both product W and product X are followed by the letter “b”.
3. Product X is significantly worse than both products U and V because neither of these products is followed by the letter “b”. However, product X is not different from product W because both are followed by the letter “b”.
4. Product Y is significantly worse than products U, V, W, and X because none of these products shares the letter “c” with product Y.
5. Product Y is not different from the untreated check (also called the “control”).
B) How should data without “A’s and B’s” be interpreted?
If the data table is just a list of averages, be very cautious about drawing any conclusions about which treatment may be better. The averages have not been subjected to statistical analysis to determine if any treatment is really different from any other treatment.
Such data show numerical (but not statistical) differences. Without any information on how different the numbers were that made the “average”, you will have no way to get a feel for the + or – (plus or minus) value that surrounds that average. These types of data should not be used to determine whether any particular treatment or practice is better than any other. They are for informational purposes only, and certainly not meant for making management decisions.
|
|
|
