Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I’m baffled even pop pulp rag “editorial” leadership wouldn’t be familiar with the famous “stats don’t mean normal exists” examples dating back to the 1950s:

No “normal” man:

“Out of 4,063 pilots, not a single airman fit within the average range on all 10 dimensions. One pilot might have a longer-than-average arm length, but a shorter-than-average leg length. Another pilot might have a big chest but small hips. Even more astonishing, Daniels discovered that if you picked out just three of the ten dimensions of size — say, neck circumference, thigh circumference and wrist circumference — less than 3.5 per cent of pilots would be average sized on all three dimensions. Daniels’s findings were clear and incontrovertible. There was no such thing as an average pilot. If you’ve designed a cockpit to fit the average pilot, you’ve actually designed it to fit no one.“

And no “normal” woman:

“Before the competition, the judges assumed most entrants’ measurements would be pretty close to the average, and that the contest would come down to a question of millimetres. The reality turned out to be nothing of the sort. Less than 40 of the 3,864 contestants were average size on just five of the nine dimensions and none of the contestants — not even Martha Skidmore — came close on all nine dimensions. Just as Daniels’ study revealed there was no such thing as an average-size pilot, the Norma Look-Alike contest demonstrated that average-size women did not exist either.”

https://www.thestar.com/news/insight/2016/01/16/when-us-air-...

On the other hand, maybe the editor of Scientific American being unaware of this is why there’s room for a not so new book, “The End of Average”:

”Yet, as Rose, Ed.M.’01, Ed.D.’07, a lecturer at the Ed School and director of the Mind, Brain, and Education Program, writes in his forthcoming book, The End of Average, from the moment we’re born to the moment we die, we are measured against a mythical yardstick — the average human — and it’s hurting everyone. That’s why with this book and through his nonprofit, the Center for Individual Opportunity, Rose is on a mission to dismantle this myth of the average and instead help the public understand the importance of the individual.”

https://www.gse.harvard.edu/news/ed/15/08/beyond-average

http://www.toddrose.com/endofaverage



Note the difference here is that your examples use multiple dimensions at once. Assuming Gaussian distributions are a good fit, most of the population will look relatively normal on a single variable measurement. It’s only on multivariate measurements where this stops being true. Though this is exactly what the mode expects. Plot a 2D cross section of a 9 dimensional Gaussian curve and you’ll see the center lose its density very quickly.


I'll add a fast guess from some earlier, solid experiences: Let R be the set of real numbers and n a positive integer. Consider R^n, the common n-dimensional space. Assume this Rn is Euclidean, that is, has the usual inner product, norm, metric, topology, and measure (area, volume, e.g., in the sense of classic Lebesgue measure* theory). With this overly detailed description of context (WHEW!), we move on to the main point:

In the R^n, consider a sphere. To be more clear, for some positive real number r, consider the set of all points in R^n with distance <= r from the the origin, that is, the point in R^n with zeros in all its coordinates.

Now for the big point: Consider the volume of that sphere and let n grow. Will discover that very soon nearly all the volume of the sphere is in a thin shell just inside the surface of the sphere. Or to be more explicit, for positive real number s a little less than r, nearly all the volume is in the set of points x where the distance from the origin d(x) is

s <= d(x) <= r

I encountered this in some analysis, I have forgotten, maybe generation of random points in the sphere independent and uniformly distributed in the sphere.

So, for the 10 or so body measurements of the pilots and the women, it's going to be tough (small probability for nearly any reasonable, non-pathological probability distribution on the sphere) to expect all 10 measurements to be close to the expectations, i.e., the center of the sphere (for people, the center will not be at the origin, that is, all n coordinates zero).

There is a counterexample: The girl I dated in high school, the most beautiful female I ever saw, in person or otherwise, was essentially perfect on all dimensions!


I agree. More details that may be helpful for the GP.

One trick to compare dimensions is to assume each one is somewhat a normal distribution, but each one has a different mean and variance. It makes no sense to add the difference from the average height in inches and the difference from the average weight in pounds.

It's better to divide each difference by the variance, to remove the problems with choosing the scale. Also, it's better to sum the squares of them, for technical reasons. Pulling numbers out of the air, it's something like

Points = ((height-60inches)/8inches)^2 + ((weigh-120pounds)/7pounds)^2 + ...

[I just invented some numbers. I'm too lazy to lookup some realistic numbers. I live in a metric country, so I may even had a bad conversion.]

And in the contest, they had 9 variables, so it's a long ...

What is the expected distribution of something like this?

If you have only one variable, then you expect that most people will get near 0 points. But each variable you add, the distribution changes, and most people will be further away from 0. There are some graphics of this in https://en.wikipedia.org/wiki/Chi-squared_distribution and get even more technical details. With 3 variables, most people will be close to 1 point, but with 9 variables most people will be close to 7 points.

With real data is more complicated, because the distributions are not normal. Also, there is correlation, and this model assumes no correlation. (For example tall people use to weight more.) So it's a very simplified model, but it's useful in some cases.

This is a standard part of any statistic course, perhaps the first or second course, but it's not cutting edge math.

>>> Before the competition, the judges assumed most entrants’ measurements would be pretty close to the average, and that the contest would come down to a question of millimetres. The reality turned out to be nothing of the sort.

Going from ideal mathematical models to real cases it's always difficult. In the contest the variables have correlation, and it's not a random sampling because the selection process have strong bias. But with 9 variables anyone that took a statistics course will expect a result like this, where nobody is too close to the average in all measurements.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: