Note that this is not some sort of unexpected effect due to humans not behaving mathematically ideally; this is what happens when your intuitions about 1, 2 and 3 dimensions are applied to higher dimensional spaces. Consider the goal of being in the middle 50% of a random value on one dimension; you have a 50% chance. But if you have two dimensions, and you want to be in the middle 50% for both of them, it's a 25% chance. And if you have 3, it's a ~12% chance. This already is counterintuitive, but when you ramp it up to 10 dimensions, it's a ~0.01% chance. That means that if you have a thousand people, only one of them (on average) will be in the middle 50% of all of them. Even the praised end state of the air force, with 9 dimensions and a required support of 90% of individuals of each dimension, implies that only 38% of individuals will be supported overall. (Granted, human dimensions aren't wholly uncorrelated.)
Great explanation. This is an example of the curse of dimensionality. An intuitive way to think about this is that each dimension you add is another “chance” for one of them to fall outside of the range.
Another way to think about it is the often-cited unit n-dimensional sphere. If you were to uniformly sample points from within this n-dimensional sphere, as n increased, the proportion of points lying near the surface of the sphere would increase.
That last bit seems to be the Crux of why this is so surprising -- being in the middle 50% on some dimension correlates positively with being in the middle 50% on the other dimensions, rather than each dimension being independent as in your calculation. It's difficult for me to reconcile that, for the 400 in the study within 5% of average height, each was a standard deviation away from the mean in some other measurement.
It’s two different effects. First being within a range is not the same as being at the center of the range. Someone 4% above average height should have ~50% of their dimensions larger than that. Similarly someone at 4% below average should have around 50% of their dimensions below that. At best someone in the exact middle of the range only has so much buffer to work with.
Second the correlation is less significant than your assuming. My legs are the same length as some people a full foot shorter than I am.
Sure, but being in the middle 10 percentile in height(or some other dimension) would serve to "normalize" the sample; so 400 people are close to the center of the larger range. Despite being close to the middle of the range, some dimension is far from the center of it's range.
Yes, the correlation is not as strong as I would assuming -- that was really the point of my comment. You are a sample size of one, so your anecdote doesn't mean much. However, based on this work, apparently almost everyone has a similar anecdote: after normalizing for height, there is another common dimension which is "unusually" large or small.
The slope of a bell curve near it’s center is almost flat. This means you end up with a fairly uniform distribution when looking at values near the median. Which makes outliers within that range more common than intuition suggests.
I find it more intuitive to to discard the spatial intuition and think in terms of events. You want something in the X% norm? Think about it like a X%-heads biased coin flip. If you flip it enough times, it'd be shocking to get all heads. Even if the coin flips (dimensions) are correlated, many coin flips (dimensions) means at least some tails (dimension outside the norm).
Should note that the study seems to be working on an extremely narrow range, the 30% in middle to quote the article. So measurements in the 35-65% percentile from what I understand.
In layman's term. It's so narrow that there are more people 1 inch off than there are people within the expected height. It's crazy.
Probability is talking in terms of standard deviations nowadays. They are selecting less than half a standard deviation, it's hyper selective. I'm curious how many people would fit the norm if the study was looking at 1 standard deviation. Surely a lot more.
For reference. Selecting the 30% on six metrics is keeping less than 0.01% of participants. Selecting the 68% (one deviation) on six metrics is keeping 10% of participants. It's night and day. Should be even more in practice because measurements are correlated.
This comment was what made this article make sense to me.
Just to add to add to the mathematical intuition here (please correct me if I'm wrong): if you're thinking of it as a unit line/square/cube then total n-dim area is 1^n, and the portion in the 50% range is (1/2)^n, where n = number of dimensions. So that should simplify to 2^(-n).
Note this works out to 10th dimension as 0.09765% or 0.1 person per 1000.
ETA: As one of the comments below points out, you can also model it as a binomial distribution.
Probability of getting all heads, given p=0.5 is (n!)/((n-k)!k!) / 2^n. Since n=k since we're looking to get all heads at all times, this also simplifies to 2^(-n).
P(simultaneously within middle 50 % of n independent distributions) = 0.5^n = 2^(-n), like my parent comment says.
But interestingly, every relevant comment here either got something wrong about the final frequency or percentage, or corrected the wrong thing. (Parent comment, grandparent, 2 aunts and 1 cousin.)
0.5^10 = 2^-10 = 1/(2^10) = 1/1024 [i.e. exactly 1 person per 1024] ~= 0.00097656 ~= 0.09766 %.
Or to work the example in the article: Being in the middle 30% of any three traits has a 2.7% chance (matching the quoted 3%). For ten traits it's a 1e-6 chance.
This is only true if each dimension is independent and identically distributed random variables, which is doubtfully true for humans. That said, the overall intuition is probably more correct than incorrect...
They are neither fully independent nor fully dependent. It falls somewhere in the middle. The same will apply, just in a slightly less dramatic effect.
But surely the engineers were who were inventing jets and rocket guidance systems and radar understood probabilities of independent events. The point, which the author almost misses completely, is that they dramatically overestimated how correlated the measurements were.
Beyond that, while there's no accounting for taste, I find it to be an appallingly bad article. It's poor man's Malcolm Gladwell.
Understanding the probabilities of independent events is a different thing than identifying them.
It is very common for engineering projects to encounter problems caused by an issue which was understood but was not identified. Most engineering failures fall into this category. Humans simply make mistakes.
Consider this set of statistics:
90% of elementary school teachers are female [1].
90% of veterans are male [2].
Unsurprising.
But, there are 2 million female veterans in the US, and only 1.7 million female elementary school teachers. That means if you talk to a random American female, it is more likely that she is a veteran than an elementary school teacher. I think a lot of disagreements on Hacker News and elsewhere stem from people saying "Well 90% of the time, X is true" without realizing that the 10% they are choosing to ignore is comprised of millions of people.
That's because you need to know that there are ~20 million veterans and ~1.87 million elementary school teachers, i.e. you need to know the base rate if I'm reversing the calculations properly.
Another flaw of averages is apparent when you realize that the average person has less than two eyes.
That must mean that well over 10% of adult male (from the other statistic that 90% of veterans are male) Americans have served in the military. I never would have thought the number was that high, especially with the military being all volunteer for decades.
There are about 2mm active and reserve military members and about 400mm Americans on the whole, so there are, at any time, 0.5% Americans in the military. Assuming the average enlistment is 1 term or 4 years, and people have about 80 year lifespans, then you get ~20 (80/4) "generations" of enlistments over a person's lifespan. 20 * 0.5% is 10%, so about what you would expect for "living veterans" if you had a constant cycle of veterans through the system. That's serious back of the napkin math (lots of assumptions!), so it might be incorrect, but I think it shows that 10% isn't too far from what you might expect, right?
Possibly, though I would think that many who serve do more than 1 enlistment. It's just that from my own personal experiences I've encountered few veterans in my life (except for older generation family members who served in WWII), that's why intuitively I find the number a bit surprising.
Yes, it should be median though the original is true for symmetric distributions. If you apply the central limit theorem, then you could say that half of all randomly samplings of a population are, on average, dumber than the population's average.
Since intelligence must be a one-sided distribution it's probably right-tailed, which means the mean is higher than the median. So I expect (slightly) more than half of the population to be dumber than average.
Those are all small, independent random contributions that naively would lead to a normal distribution per the central limit theorem. But it can't be quite a normal distribution due to the lower bound of 0 (in terms of absolute intelligence, not IQ). A better candidate would be a log-normal distribution which is always right-skewed.
Exactly half under is correct only for if none is exactly at the median, which requires (but is not necessarily the case when) there are an even number of measures.
This is midly-nitpicky, but those numbers only include public elementary school teachers. There are a lot of private/parochial school elementary teachers out there. There might very well be more female elementary school teachers then female veterans.
I would argue that Bayes Theorem is taught in high school basically conditional probabilities and inferences. The issue is that applied Bayes Theorem is not taught. What do you do with information given with the data at hand?
Bayesian updates should be taught in schools, at least at the most coarse-grained level. Not to let people calculate anything; almost nobody is going to need that in life anyway. But so that some authority (which a school is) tells people they're allowed and supposed to update their beliefs on evidence. That evidence drags belief more in one direction than the other. That 0 and 1 are not probabilities in real life. That being certain of something is a rare thing, that they should embrace being more or less sure. That this is not someone's random worldview, but there is a proper (and rather fundamental) mathematical formalism behind it, it's just impossible to apply it fully in real life, so we have to approximate it.
> ...the 10% they are choosing to ignore is comprised of millions of people.
I don't understand your point here. Is it that somehow, some people are more than 10% likely to run into the 10%? Well, that's obvious. Roughly speaking, you'd expect 10% of commenters to be in that 10%, unless we're talking about something that disproportionately does or does not affect people interested in technologies, startups, etc..
The point is that people often have an opinion, cite one statistic to validate their opinion, miss the bigger picture (because summarizing anything complex with a single number, rarely tells the whole story), and in the face of additional evidence stick to that one statistic that validates their original opinion. I see it happen a lot. Both on HN and elsewhere too. Presenting multiple statistics that shine a different light on the same population, enriches the conversation, and gives a more complete characterization of the population/situation/phenomena etc.
I also personally found that combined set of statistics enlightening because I previously did not realize how large the population of female veterans is, since I had only read about the percentages not the absolute numbers.
It's a problem of lack of STEM education, unfortunately. More specifically, of metrology.
"90% of elementary school teachers" is not directly comparable to "90% of veterans". You have to multiply them by their respective populations to get numbers that can be compared. In general, a thing of the form "$amount of $something" is not comparable to "$amount of $somethingelse". That amount being a fraction with a '%' sign in front of it doesn't change anything.
Unfortunately, schools aren't beating people with sticks until they internalize that point, while at the same time a good part of sales and marketing relies on people not being formal enough in their thinking.
Yes, I certainly wish schools would teach this better. I think showing different examples with surprising conclusions/outcomes helps people internalize that point.
> Out of 4,063 pilots, not a single airman fit within the average range on all 10 dimensions.
> Less than 40 of the 3,864 contestants were average size on just five of the nine dimensions and none of the contestants — not even Martha Skidmore — came close on all nine dimensions.
This seems more an issue of the "curse of dimensionality"[0] more than the "flaw of averages". Be very wary when you are trying to draw conclusions from a dataset with many dimensions.
> Even more astonishing, Daniels discovered that if you picked out just three of the ten dimensions of size ... less than 3.5 per cent of pilots would be average sized on all three dimensions.
FTA: ”These formed the dimensions of the “average pilot,” which Daniels generously defined as someone whose measurements were within the middle 30 per cent of the range of values for each dimension.“
0.3³ = 0.0027 = 2.7%, so if those measurements are independent of each other, it isn’t surprising that he found “less than 3.5 percent”.
Human dimensions are closer to a normal distribution. (Not quite normal, but let's pretend.) In that distribution then 46% of subjects should be in the middle 30% of 3 dimensions. [0]
[0] With 4000 subjects they presumably have all sizes within ~3.5 stddevs represented. The middle ~1.2 stddevs should hold ~77% of the population. 0.77^3 = ~46%
> Human dimensions are closer to a normal distribution
Human height is (it's the textbook example of a real, intuitive_that is, using the normal linear scale—physical measure that is a good fit for a normal distribution) plenty of other human dimensions are not.
> Human height should be a negative binomial distribution, since human height can't be negative.
That's a sensible analytical speculation, but it empirically fits very well with a normal distribution.
The fact that zero is, for adult male height (the typical cited example for fitting a nor Al distribution, though adult female height also works), around 17 standard deviations below the mean helps: I won't bother to calculate how little should be below that, since below 7σ in a normal distribution is 1/780 billion.
Been noted here on HN: the average web form excludes lots of people. Lots of folks cant fill in: First/middle/last name; unique username/password; social security number; street address; bank account.
And not just folks in Micronesia or whatever. Heck my business partner hasn't got a deliverable street address - no mailbox at his house. He has a PO box. Is a nightmare getting Amazon deliveries that aren't lost, or left in the bushes, or returned undeliverable.
The name field situation has never bothered me as a typical American with the standard name format, but as I've grown and met people from different cultures I've learned just how frustrating that part of forms can be. Even (or rather, especially?) government forms are terrible about this.
Even typical Americans struggle with this sometimes if they have, say, hyphenated last names. I've encountered plenty of software out there that will reject or fail on even such a common case as this.
There's so many times that an ostensibly international website has rejected me because I don't have a US-formatted numerical zip code. Even when they have a country field.
I remember when this idea came around Hacker News the last time. I mentioned this over lunch, and one of my (highly respected and deservedly considered intelligent) younger coworkers called BS on it. His object was: "Why would a government build an expensive, high tech piece of equipment, then not ensure it could fit the operator?" He just couldn't imagine it.
Between 2020 and WWII, there is a huge difference in available technology and scale. There were a lot more planes and a lot more pilots back then. It's practically a WWII trope: Arguably "inferior" weapons systems win out because they can be produced in larger numbers with better maintainability and support logistics. (1)
Depending on how you evaluate it, the amount of firepower and capability embodied in one WWII fighter is greatly dwarfed by that in one gen 4 or gen 5 fighter jet.
(1) (Though, in WWII, Allied planes often had a tremendous advantage in performance, largely because they had access to far better fuel. This allowed for much higher compression ratios and higher pressure superchargers, so they could produce more power, more efficiently, for less weight. "Greg's Airplanes and Automobiles" on YouTube covers this.)
>Why would a government build an expensive, high tech piece of equipment, then not ensure it could X
Maybe short design periods in some cases too. Either way it was not too uncommon. There were bombers with escape hatches big enough for the crew, provided they didn't wear parachutes[1]. Fighter bomb releases that required ducking down to reach them[2] and fighter planes who's fuel feed stopped when pulling negative Gs [3]
Not to mention desperation moves like the british STEN sub-machine gun which was basically what you get if you say "I need 800,000 of these and they just have to meet these criteria, eject everything else".
The Sten was a beautiful accomplishment. Being cheap, simple, optimized for quick mass production, and just good enough to get the job done is a perfectly legitimate set of requirements. It’s like the IKEA LACK.
> Why would a government build an expensive, high tech piece of equipment, then not ensure it could fit the operator?
Why would they send a $250M probe to Mars, and not ensure a consistent set of units is used?
I mean, putting the above two examples aside, one could list plenty of much bigger screwups by the government. Why is this person assuming a competent government?
I wish I knew the name for this fallacy, but it is essentially a restatement of "I can't think why X would behave this way, so it must be because Y." Or as I tend to call it: "Out of ignorance, comes certainty". I mean, you just admitted you didn't know! How are you drawing conclusions?
I am sure you can also produce a long list of screwups by private companies. Assuming you made an honest effort of course. Private companies fail and go bankrupt every single day. 90% of private companies fail within a year. The real problem is not government/private. The real problem is making a large number of people work well together while making sure millions of decisions are all 100% correct. That is a very hard problem. What is surprising to me is that large organizations (governmental/private) sometimes succeed. That is the real miracle.
> "Why would a government build an expensive, high tech piece of equipment, then not ensure it could fit the operator?"
Sometimes people just do things wrong. I imagine there’s some name for the fallacy of assuming all humans are perfectly intelligent, rational actors.
Why wouldn’t they do it? Because the people in charge of the decision didn’t think it was that important. That’s all it takes. And to their credit, making airplanes adjustable is harder than it sounds. A fixed seat is much lighter and cheaper than an adjustable one.
But, it turns out the trade offs are worth it. This is sort of like “never attribute to malice what can be explained by incompetence”, but “incompetence” is too strong of a word. They just didn’t know.
Our technology also continued to evolve throughout the war, updated fighters, better engines, even whole new designs. The Germans as far as mass produced designs went were effectively frozen in 1939. A lot of this was the complete inefficiency of the Reich Air Ministry at project management, as well as the political supremacy of the Heer over the Luftwaffe. In addition to the political whims of a dictatorship (Technological choices were often driven by ideology, and perception, much more than in the other dictatorship at the time (the Soviet Union, where for the most part, the technocrat reigned supreme))
All this said, from purely a material point of view the war in europe was won by the allies in 1941 when the US joined the conflict, but the Axis could have dragged the conflict out much longer had they had an effective strategic bombing program, and more modern fighters.
Regardless of Spitfires, Alan Turing, anti-sub technology, RADAR etc, WW2 was won and lost on the Eastern front. The Normandy invasion would have been impossible without the Russians sacrificing between 8-11 million soldiers. This compares to about 400k each for the UK and US.
I'm just saying, don't get carried away with the technological superiority argument, there was plenty of innovation on both sides. After all, the engineer that designed the Apollo rocket was a Nazi weapons scientist.
No real disagreement there - the war was won on the back of Russian Bodies, American Trucks, and British Grit.
The germans did indeed have wonderful technology in labs, and had it made it into mass production, it would have made the war stretch out much much longer.
Our technology also continued to evolve throughout the war, updated fighters, better engines, even whole new designs. The Germans as far as mass produced designs went were effectively frozen in 1939.
Well, not completely. "Greg's Airplanes and Automobiles" has a nice video on, "Why was the BF109K faster than the P51D? MW 50!" So to try and compensate for only having lower octane fuel, German engineers in WWII went to water/methanol injection for emergency power. Things did go back and forth quite a bit. Granted, the BF109 was one of the most advanced late 1930's designs going up against 1940's allied wartime designs. But there were times when updates on older designs would outclass the other sides slightly older versions, and this happened repeatedly on both sides.
Technological choices were often driven by ideology, and perception, much more than in the other dictatorship at the time
"Yes, but can it dive bomb?" probably did about as much to compromise designs of Nazi fighter/interceptors and slow the release of new airplanes as espionage and sabotage by the allies!
The end of the war began when Germany invaded Russia. Europe/Russia had already killed 30+ million German soldiers and destroyed most of the German war machine when the US joined the conflict. The real contribution of the US was stopping Russia from invading Europe after the war (see Berlin Airlift) by threatening to drop nuclear bombs on Russia (Russia at that point wasn't a nuclear power).
This reminds me of an old math joke. A statistician and a mathematician went duck hunting. From a blind, the mathematician shot and hit a duck. Their retriever went and got the duck. Then the statistician spotted and shot at another duck but shot too high. Quickly the statistician shot again but too low. The statistician then beamed with pride, on average, that's a dead duck.
This reminds me of a Planet Money podcast where they try to figure out the characteristics of the most typical American, or as they put it "who is the person I'm most likely to run into?"
To construct this archetype, they used the mode of various dimensions rather than the median or average.
The USAF employs averages in all kinds of interesting ways!
"How US nuclear force modernization is undermining strategic stability: The burst-height compensating super-fuze" (or: How I Learned to Stop Worrying About Undershoot and Love Terminal Detonation Timing)
Human Dimension and Interior Space [0] is an interesting book if you're into this sort of thing. I mean, it's actually mostly not. It contains a vast number of tables covering percentiles for various human dimensions broken down by sex and age group.
It does, however, also contain diagrams showing what measurements are relevant to certain activities. Chairs for instance, are covered for eating, office work, and lounging. Fun fact, the seat height of a chair is properly called "popliteal height".
I end up pulling it out to answer such questions as: "Can I make the apron on this table go any lower assuming chairs in a normal-ish range of popliteal heights while preserving enough thigh clearance."
I wonder if this applies to something like performance reviews where measuring overall performance as something relative to 'average' inevitably doesn't make sense to anyone as perhaps there is no average there for a job with any level of complexity.
The military is one of the few governmental organizations where reality is ruthless enough that they are forced to adapt to it, rather than distorting reality to fit their agenda.
The referenced study of human dimensions was not the only thing done by the military to improve aviation safety. The discovery of human factors in user interfaces was also recognized. There had been some standardization of the cockpit controls of different aircraft during WWII and before, but not so much for the instruments and displays. That was no easy task, but the problem was recognized and addressed.
Not to pick on a single stereotype but it seems we've become memes and it's led to an attack of the clones on the ideas of diversity and inclusion flipping everything on its head
Think about how 'fitting in' is also a term for 'averaging out' and ask yourself how much we value mediocrity?
You can only join our group if you fit within one of these narrow guidelines and wear one size fits all clothing
For crying out loud I'm sick of all the fakeness in the name of technology
There's something more impersonal and less about authenticity, when you look at the factors involved in joining a group and becoming part of its identity, that's for sure.
For that same reason though, I'm not sure I'd say that other groups differ so much. "Fake" realtors, "the typical hiker," or whatever it may be. The in-groups can often tell you about it first.
Groups do seek their own identity over time, and sometimes groups can have identity crises, in which they arrive at a consensus on a new set of values after some other set of values has run its course.
Even more importantly, individuals encountering this perspective--before the group does--have the opportunity to run early experiments and find new ways of being or communicating from the heart that may also be attractive to the social organism (of tech, or whatever).
Someone with your outlook may be able to help with this...and that could really add up to a lot.
This is a really interesting article. The most fascinating part is to learn that the AF actually listened to the man and changed their policies.
This kind of thing can happen even when only measuring a single dimension, if the distribution is multimodal. If everyone is either really short or really tall, then nobody will be near the average height.
I realize that the audience is quite different, but it’s interesting to compare the response to “no woman matches our expectations” (means they’re fundamentally broken) vs “no man matches our expectations” (means the model is fundamentally broken).
Huh, the conventional story is that the UN forces won the air war in Korea because of better plane ergonomics despite Migs having better theoretical performance. But this report didn't come out until the war was mostly over. The conventional story emphasizes things like bubble canopies but still.
>The Aero Medical Laboratory hired Daniels because he had majored in physical anthropology, a field that specialized in the anatomy of humans, as an undergraduate at Harvard. During the first half of the 20th century, this field focused heavily on trying to classify the personalities of groups of people according to their average body shapes — a practice known as “typing.” For example, many physical anthropologists believed a short and heavy body was indicative of a merry and fun-loving personality, while receding hairlines and fleshy lips reflected a “criminal type.”
Can anyone point me in the direction of more information on this? Curious to learn more but my searches didn't turn up anything related.
They make the leap to "better-fitting cockpits solved the problem of air crashes" without really proving it. I understand their theory, and it sounds believable. But it might be a spurious correlation.
By the way, this is also the reason why things like UI design have gotten so bad: Most Machine Learning algorithms fit averages. Combine them, and you learn surprisingly little!
This interesting but all a bit wordy and meandering.
Tldr: The air force was using the average dimensions of their pilots to design aircraft. Coincident to this, there was a large number of downed planes questionably attributed to "pilot error" but that many believed to be from an unknown cause. A recruit from Harvard charged with collecting data on pilots realized few people were close to the average, and recommending cockpits be fit to each individual. The AF took the recommendation and eventually companies produced aircraft with adjustable seating and cockpits.
