editorInvest Like the Best, Notes, Podcasts
8 Minutes
Link: http://investorfieldguide.com/epstein/
About David: Best-selling author of The Sports Gene and Range: Why Generalists Triumph in a Specialized World. A former journalist at Sports Illustrated and ProPublica, David is also known for his talks on performance science and the proper use of data across many fields including sports, medicine and natural sciences.
Transcription: Otter.ai
Epstein’s Research Process
- 10 journal articles a day for 1 year; hire translators for foreign journals
- Consults with statistician
A weaker 10,000 hours idea (Tiger vs Roger Problem)
- Contrary Research Favoring Breadth
Showed elite athletes did not require a head start in deliberate practice. More likely specialization was delayed. A long sampling period exposed them to many sports which allowed them to better match their abilities to the sport. Evidence: Success of Olympic talent transfer programs in other countries
Why?
Lots of variation in how people respond to stimulus. True of medicine. True of training. You baseline ability is uncorrelated with your ability to improve with training, which makes extrapolating difficult. “So much to gain from fitting people into the right sport”
- Supporting research flaws
- “Restriction of range” problem with the study of 30 violinists. When you squash the range of a variable that is correlated with the dependent variable you risk understating the correlation with the restricted variable. In this case, the sample was violinists who had already been accepted to a famous academy. We have squashed their innate talent even though it likely has a wide range. Likewise, if you studied the correlation of height to points scored in basketball for NBA players you find a jarring negative correlation but that is because you are selecting from a sample of abnormally tall players, to begin with. You’ve squashed the height variable, which would lead people to think that height has no impact on points scored.
- Inconsistent numerical data, no estimates of variances on variables, poor statistical inference
Learning
- When to be like Tiger?
- Kind learning environment
- Fast, accurate feedback
- Discrete turns
- Well defined rules
- Kind learning environment
- When to be like Roger?
- Wicked learning environment
- “Martian Tennis”: You see people out there playing, something’s going on, you don’t know the rules, it’s up to you to introduce them. And they could change at any moment without notice. And that’s the situation that we’re actually in for most of the things, the complex things that most of us care about.”
- Wicked learning environment
- Most surprising study in Range: air force study is a natural experiment. Professors who were the best at causing students to do well in their own class do well on the test, (ie overperforming compared to the baseline characteristics they came in with) systematically undermined those students “deep learning” (performance in the follow on courses).
- Professors taught narrow performance to optimize for their own exam to their detriment for overall learning. They undermined the students “making connections” framework. Professors failed to learn themselves because the students who would feel rapid progress would rate them highly. “really wicked feedback”
- Professors themselves are incentivized to maximize for short term evaluations which have impaired their ability to teach frameworks that students can apply in novel situations.
- Professors who did not teach to the test taught broader concepts relying less on “using procedures” knowledge. This type of knowledge is most effective in kind learning environments where possible tasks and choices are restricted.
- “Closed skills”: techniques that you can teach very quickly and see an advantage. these are temporary advantages as people with broader frameworks eventually catch up but have brought wider understanding as well.
- Around the world, we are performing better on “culturally reduced tests” (meaning tests that are not influenced by formal learning). Our collective performance should stay stable on this portion of tests but in fact, our performance is increasing. Known as the Flynn effect. Flynn speculates “we have moved to a world where we are used to classifying things to grouping things instead of being stuck with lots of concrete knowledge and, and factual knowledge.” Pre-modern people did not have much need for classification, but the modern world relies heavily on this ability sincewe’re constantly laterally translating knowledge to different areas we’ve never seen. This ability to have knowledge that we don’t have from hands-on exposure is really important.
(Me: Don’t be fooled by a sense of progress when the task you are excelling at is not varying. Being able to match abstract models to a correct strategy is a more valuable goal and benefits from practice in dealing with variation. )
- Learning hacks supported research but ignored by the media (3 out of 5)
- Testing: Test people before they have a chance to study. It primes your brain and exploits the “hypercorrection” effect — our tendency to remember the correct answer to a question you tuned out to be wrong about
- Spacing: Intervals between practice make learning stick longer. A useful technique is to learn several subjects at once. Switching provides natural breaks.
- “Difficulty isn’t a sign that you’re not learning but ease is”. To maximize stickiness you actually want to re-learn something just after you have forgotten it! Your steepest learning occurs when the task is difficult.
- Interleaving: Mixing types of problems will extend the time it takes to learn one type but improves broader ability to match approach to the type.
Grit is Misunderstood
West Point study: the survey which measured grit was more predictive than the conventional metrics for predicting who would complete Beast Barracks (physically demanding module of training). This grit survey was applied to other domains like the Spelling Bee championship contenders. Grit appears to have a measurable, effect independent of other variables.
- Problem with these studies is they suffer from the same “restriction of range” problem
- The measured effect is significant but small. Much smaller than what companies are interested in testing for.
- Sample of people is dedicated to a short term task like winning a spelling bee or completing their training. Very difficult to generalize to a wider measure of this individuals’ determination when the task is less well-defined
- When zoomed out, we find that attrition is a poor proxy for ‘lack of grit’. Attrition is occurring in a time when people in these studies are going through periods of rapid self-discovery and personality change during their early 20s (this is the peak change period in our lives) and re-assessing as they search for “match quality”. The degree of fit between work, interests, and ability.
- Grit is not necessarily stable. It seems to vary within the same individual depending on the context or task.
- In general, the study of grit is has been contained to very short term, narrow environments
Avoiding Premature Optimization
Paul Graham admonishes against working towards some projection of future self when you are young since what you can conceive is too limited because your experience is limited. Too risky to throw yourself on a path based on such a limited hunch.
- Our personality is only .23 correlated between teen years and middle age
- We learn by doing then reflecting, rather than introspecting to form a theory about ourselves. Frequent trial and error is a better way to decide which direction to go.
- Harvard’s Darkhorse Project studies how people match careers. The students who matched best excelled in short term planning.
- Economist Robert Miller refers to the “2 arm bandit process”. Metaphor on a gambler pulling levers in a casino, getting feedback, before focusing on a game. He advocates jumping into high risk, high reward fields early because you learn the most from them. That informational signal is a faster input into your decision path.
Opportunity to recombine
Information including specialized information is disseminated more widely and quickly than ever and at an increasing rate giving people greater opportunity to recombine from all the available information.
- Parallel trenches: “everyone’s in their own trench and not usually standing up to look over at the next trench even though that might be where their answer is” (this is why he hires translators)
- Gunpei Yokoi — Nintendo employee who used lateral thinking to recombine older, cheaper, “withered” technologies to create products including the GameBoy. The GameBoy competed with more advanced products on the basis of its ease and durability.
- Yokoi viewed cutting edge technologies as zero-sum arm’s races fought by specialists. “Many more opportunities to take this stuff that was already well known that everyone was looking past and recombine them in new ways”
- We are in an age where its feasible for a generalist to crowdsource specialists in novel ways which allow them to outperform specialists themselves (Kaggle has been able to solve problems that have stumped NASA)
- Specialists perform better when the next steps are clear and the path is more obvious. The right mix of generalists and optimists depends on how well characterized the problem is.
- 3M has many interesting examples and lateral thinking is entrenched in their DNA. They maintain a “periodic table of technologies” so its teams can use their awareness to recombine.
- Superman or Fantastic Four
- Metric that best predicted a comic book creator’s potential to write a blockbuster was the range of genres they covered, not reps or experience.
- In addition, they found that a team of writers with combined experience in diverse genres outperformed a single writer unless the single writer was fluid in at least 4 genres. “Individual in some ways is the best unit for integrating information” although a diverse team is next best.
- To a specialist with a hammer “everything looks like a nail”
- Specialists continuing to administer procedure in face of evidence that it doesn’t work
- Scandinavian meniscus placebos undermine the benefit of surgery
- Practices that make intuitive sense (“bioplausible”) but poorly supported by evidence
- When outcomes are poor surrogates for health: stents for otherwise healthy people with a narrowed artery do not reduce their heart attack or mortality rates. A wider artery is not a perfect proxy for the desired outcome because “There’s a clogged artery, how could opening it up not work. It’s got to work except it turns out the body’s much more complicated than like a kitchen sink, and we didn’t design it. And it’s the disease is much more diffuse.” (Me: any counterintuitive but effective remedy that works by using a seemingly oblique strategy is at risk of confusing surrogate markers for the outcome. Hormetic processes, body’s use of iron, etc).
- Specialists continuing to administer procedure in face of evidence that it doesn’t work
- A better way forward
- Need generalists to work with the specialists for a more zoomed out view which better aligns practice with objectives. Medicine seems especially prone to the errors and resistance to reform that can result when an inordinate amount of specialists populate a “wicked” learning environment
- Medicine and similar “wicked” environments are “devilishly” hard. It will take generational change as the entire approach to “how information is evaluated and how scientific thinking works”. Need to de-specialize a bit and increase breadth. Statistical understanding requires more than “hitting buttons on a statistical program”
- Freeman Dyson has said we need more birds in medicine. “Frogs are down on the ground looking at like a very narrow area of the ground, the birds are up. They don’t have a good definition on the ground, but they see the bigger picture. And I think we need to make the medical ecosystem more friendly to some of these birds who are looking at the outcomes we actually care about, not just those surrogate markers or did I fix the meniscus?”
Published
