You can't have paradigm shifts by following the paradigm.
How I think of it is we need a distribution of people (shaped like a power law, not a normal).
Most people should be in the main body, doing what most people do. They're probably the "most productive".
Then you have people in the mid tail who innovate but it's incremental and not very novel. They produce frequently (our current research paradigm optimizes for this). But there aren't leaps and bounds. Critically it keeps pushing things forward, refining and improving.
But then there's those in the long tail. They fail most of the time and are the "least productive". Sometimes never doing anything of note their entire lives. But these are also the people that change the world in much bigger ways. And sometimes those that appeared to do nothing have their value found decades or centuries later.
Not everyone needs to be Newton/Leibniz. Not everyone should be. But that kind of work is critical to advancing our knowledge and wealth as a species. The problem is it is often indistinguishable from wasting time. But I'm willing to bet that the work of Newton alone has created more value to all of human civilization than every failed long tail person has cost us.
In any investment strategy you benefit from having high risk investments. Most lose you money but the ones that win reward you with much more than you lost. I'm not sure why this is so well known in the investment world but controversial in the research/academic/innovation world.
Hah. I think of it as a slime mold. There's the main body (bodies?), but it's always shooting out little bits of itself that try weird stuff - founding underwater communes, or climbing mountains in Crocs or something. Most of these offshoots don't have that much of an impact, but occasionally one lucks out and discovers America or peanut butter and the main body saunters off that way.
Yeah we find this type of optimization all over nature. Even radiation is important to create noise. We need it in machine learning. A noisy optimizer is critical for generalized learning. Too much noise and you learn nothing but no noise and you only memorize. So there's a balance
I'm reminded a bit of a robotics professor I had at uni, a very long time ago in the 1990s. When he was at uni in the late 70s he and a couple of friends were working on a robot arm, which didn't work because it kept dropping stuff. They made a more precise linkage for the gripper, and if anything it was worse. They made ever more precise bushings for the pivots, and it just didn't help or made it worse.
Eventually after a conversation in the pub with one of his friends who was studying sports physiotherapy, they ripped out the 47th set of really precise little Teflon bushings and put in new ones made of medical silicone rubber tubing.
Now all the joints were a bit sticky and squashy and wobbly, and it picked everything up perfectly every time.
I get the "outliers are useful" thing you're trying to emphasis. But as someone from a mountainous country, please dont "climb[ing] mountains in Crocs", we regularily get media reports of hopelessly underequipped people having to be rescued with a whole team of people, in the middle of the night, with horrible weather, usually also endangering the people that do the rescue. I guess what I am trying to say is, there is a limit to how silly you can/should be.
What is the expected value, of some dude at university spinning dinner plates in the cafeteria? What a silly pointless thing to do! Of course, if you're physics Professor Feynman, you get a Nobel out of it, so do the silly pointless things after all!
If the bongo man taught me anything it's that you should do something for the pursuit itself. The utility can always be found later. But chasing utility first only limits your imagination
I don't understand how this is a power law and not normal. The "long tail" is usually mentioned in a normal distribution being the right-most end of it.
I think you have a misunderstanding because a heavy tail is essentially one that is not exponentially bounded. A long tail being a subclass.
There's kinda a big difference in the characteristics of a normal distribution and power and I think explaining that will really help.
In a normal you have pressure from both ends so that's why you find it in things like height. There's evolutionary pressure to not be too small but also pressure to not be too large. Being tall is advantageous but costly. Technically the distribution never ends (and that's in either direction!). Though you're not going to see micro people nor 100' tall people because the physics gets in the way. Also mind you that normal can't be less than zero.
It is weird to talk about "long tail" with normal distributions and flags should go up when hearing this.
In a power distribution you don't have bounding pressure. So they are scale free. A classic example of this is wealth. It's easier to understand if you ignore the negative case at first, so let's do that (it still works with negative wealth). There's no upper bound to wealth, right? So while most people will be in the main "mode" there is a long tail. We might also say something like "heavy tail" when the variance is even moderate. So this tail is both long and the median value isn't really representative of the distribution. Funny enough, power laws are incredibly common in nature. I'm really not sure why they aren't discussed more.
I think Veritasium did a video on power distributions recently? Might be worth a check.
Before I watched this I would default to thinking about most distributions as normal. It's really fun to think about whatever "game" you are playing - wether you are building a business or trying to win a carnival game - and consider if the results follow a normal distribution or a power law?
How I think of it is we need a distribution of people (shaped like a power law, not a normal).
Most people should be in the main body, doing what most people do. They're probably the "most productive".
Then you have people in the mid tail who innovate but it's incremental and not very novel. They produce frequently (our current research paradigm optimizes for this). But there aren't leaps and bounds. Critically it keeps pushing things forward, refining and improving.
But then there's those in the long tail. They fail most of the time and are the "least productive". Sometimes never doing anything of note their entire lives. But these are also the people that change the world in much bigger ways. And sometimes those that appeared to do nothing have their value found decades or centuries later.
Not everyone needs to be Newton/Leibniz. Not everyone should be. But that kind of work is critical to advancing our knowledge and wealth as a species. The problem is it is often indistinguishable from wasting time. But I'm willing to bet that the work of Newton alone has created more value to all of human civilization than every failed long tail person has cost us.
In any investment strategy you benefit from having high risk investments. Most lose you money but the ones that win reward you with much more than you lost. I'm not sure why this is so well known in the investment world but controversial in the research/academic/innovation world.