We use averages all the time without really thinking what we are doing. We believe in them and worst, we take decisions based on them. But most of the time averages can mislead you as this famous joke:
Once upon a time there was a statistician with the head in a freezer and the legs in a fire. A person worried that he was in danger to die asked him: how do you feel? He answered: in average I am fine.
Unless you have data that is pretty homogenous, averages are not enough. For example parents are obsessed with their kid’s grade average. What is better: A- or B? Well, as most things in life, it depends. If the A- is a kid with all As and one D, I guess I would prefer my kid to have all Bs!
So, when you look at uni-dimensional data, e.g. school grades, you need to look more:
- The maximum and the minimum: this will tell you the spread of your data and rapidly point out to outliers, data errors, or missing data.
- The median: the value that splits your data in two sets of the same size. Many times this value is more meaningful than an average.
- The most frequent value: this is called the mode and makes sense if you are dealing with a finite set of values (that is, you have a discrete distribution). The most common value may point out to another data issue (e.g. a default value).
If you know a bit of Excel, the best would be to plot the distribution. For that is important to bucket well the data, so you can get a nice histogram. Beware here that bucketing data using equally space segments of the possible range of values only makes sense for uniform distributions. Otherwise, the buckets should have the same volume. That is, each bucket has the same number of samples and then you can take the average of those values as the value for that bucket.
Let us look at an example. Consider the grade sets for two students, A = {5,5,6,6,6,6,6,6,7,7} and B = {3,3,3,3,5,5,8,10,10,10}. I give them ordered for simplicity, but in general data will come in any order. Notice also that the order can be different for each student, so we are comparing all the grades as a whole and not the grade in each subject.
Well, A and B have the same average, 6, however B is failing 4 subjects while A is approving everything. The minimum will point out this difference right away. The maximum on the other hand tells us that B, on the other hand, excels in some subjects, so he might be very successful in them (however our school systems force all kids to have a minimum to pass to the following grade, why?). The median of A is 6 while the median of B is 5, so in this case seems better than the average. The mode for A is 6 while for B is 3, showing even more the
differences.
One way to plot the distribution is a direct graph of the ten values as:
Here we can see that A is above in 6 values and below in 4. However as the order might be different, we can only say that A is above in 4 values because there might be two ties (5). One solution would be to order the data of B by the order induced by A (that is, the subject order of A, sorted by grade). Another way to see this is to do a histogram that counts the number of subjects with each grade like this:
This histogram highlights the mode of each student (the tallest bars in each color). Here clearly B has a lower grade in 4 subjects and a higher grade in the same number of subjects.
When we have too much data to understand, we can collapse data in buckets and here is important to have the same number of values per bucket to be able to compare fairly the two data sets. For example, if we use groups of two values we obtain the following side-by-side comparison:
gummygunk
andulkaphoto
vaporwave
dededos
snowghoul
tangentwithtanner