Tetlock, Dunning, and Kruger 


First, Tetlock, from Lamb’s Reign’s Hyper-Credentialism & The Tetlock Effect by Jordan Wilson

—<Quote begins>—

Phillip Tetlock is an expert (forgive the irony) in how wrong experts can be across an array of professions and arenas. He understands the problem of pseudo-experts but also understands the problem of arrogant “actual-experts” who make a mess of everything. After a long term, twenty-year study following 284 experts, making over 80,000 forecasts according to their field of expertise, he found that the predictions made by “the experts” were no more accurate than what the New Yorker called “dart throwing monkeys.”

Even further, according to this Time article,

University of Pennsylvania professor Philip Tetlock has shown that nonexperts typically make better predictions than subject matter experts in a variety of domains—they are better able to draw upon an eclectic array of perspectives to solve complex problems.

It goes on:

It is this multidimensional complexity of modern day problems that diminishes the salience of singular expertise. Contemporary challenges like income inequality, climate change, healthcare access and policing strategy cannot be solved by narrow technical specialists, each swimming in their own lanes. To meet these challenges, specialist expertise is often necessary but certainly not sufficient.

Tetlock is quoted as concluding;

“In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals—distinguished political scientists, area study specialists, economists, and so on—are any better than journalists or attentive readers of the New York Times in ‘reading’ emerging situations.”

The folly of the specialist experts extends of course to the world of health and medicine. This is a world where a Johns Hopkins study suggests that the third leading cause of death in America is medical error. Let me repeat. The third leading cause of death. In the wake of the failure of the models of Professor Neil Ferguson who’s opining so influenced the worldwide response to COVID (now admitting that yes, his predictions were off), it is all the more pertinent that this folly be recognized.

Bloodletting, Lobotomies, thalidomide, the war on dietary fat, Vioxx, smoking, the food pyramid, daily aspirin, no peanut butter for babies, etc… historical instances of the overturned consensus of experts in the medical field are legion. But expert/specialist problems are also endemic in terms of the systemic problems that exist in healthcare, especially modern Western medicine. As this piece in the Atlantic outlines, the effect that specialization has had on the quality of medical care that patients with multiple co-morbidities experience is significant.

These patients, said to have multimorbidity, see a different specialist acting in isolation for each condition. Additionally, because clinical care guidelines and randomly controlled research trials typically focus on patients with only one disease to avoid confounding variables, specialists rarely know how treatment they administer interacts with other concurrent treatments. This fragmentation results in frequent adverse reactions to drug combinations, redundant or ineffective care, and overall poor health outcomes.

Many times, someone who perceives themselves as an expert because of their actual expertise in one area begins to experience “expert creep” whereby they begin to regard their expertise more expansively, as documented in this study resulting in many thousands of deaths every year.

Dr. Srivastava lays it out even more starkly in this piece in the Guardian. Our focus on specialization and siloed expertise (and the turf wars that go along with it) has evolved so far that we no longer have the ability to service the whole patient.

…why is it that the modern physician prefers to manage just one organ and a surgeon wants to operate on one type of bone? Why is it that the narrower the specialty the greater the prestige when the average patient with multiple chronic conditions desperately needs someone to heed the big picture? And what happens to all those nurses, residents and trainees who find themselves caught on the horns of a dilemma – whether to blindly follow a subspecialist’s lead and treat the organ in isolation or listen to their instincts and care for the human being? I am afraid it has become increasingly challenging to do the latter.

At the end of the day, it is the patient who suffers from lack of quality medical care, and then, in the midst of all of this, specialists within the system sit back and turn up their noses at regular people who look for better wholistic care in alternative medicine. Like anyone who would not want to be a part of this again, even after a horror-story experience is some kind of whacko-idiot.

Specialists are good and necessary, but we have come to the point in time where overspecialization, especially paired with a willingness to enact coercion based on the credentials that buttress the specialization, is a problem. When the perspective of a systematic thinker who is a non-expert is preemptively dismissed on a given issue for not having specialized credentials, you know you’re dealing with a potential Tetlock effect victim. We now face the opposite problem; we are lacking for well-informed, systematic thinkers. With the rise of radical two-kingdoms theology, which tends to scold pastors for commenting on medicine, foreign policy, economics or anything deemed “outside their lane,” the church is not immune from these problems.

According to Indian philosopher and theologian Vishal Mangalwadi’s biography on William Carey, Carey was a botanist, industrialist, theologian, missionary, educator, medical humanitarian, and moral reformer. According to Mangalwadi, Carey “did more for the transformation of the Indian subcontinent in the nineteenth and twentieth centuries than any other individual before or since.”

Pastors should take great care when they wade through varying subjects to ensure they are not being reckless or speaking in ignorance. They should not require of themselves that they be “experts” in every single topic.

But the pendulum has swung too far the other way. Now, many pastors seem to bask in the safety of the false comfort that they need not bother themselves with, that is, doing the difficult work of research in matters of complexity. Can you imagine right now how many modern seminarians might be chastising the next William Carey for not staying in his lane and sticking to what he knows, the ministerial and sacramental duties of the clergy?

—<Quote ends>—

As for the Dunning-Kruger Effect…

Let David F. Coppedge get the point across, in How to Turn Dunning-Kruger Inside Out.

—<Quote begins>—

Psychologist Andrew Danvers wrote in Psychology Today (Dec 20, 2020), “Dunning-Kruger Isn’t Real.”

The Dunning-Kruger effect is commonly invoked in online arguments to discredit other people’s ideas. The effect states that people who know the least about a topic are the most overconfident about that topic while people who know the most tend to be more humble and accurate in their self-assessment. It seems intuitively right, and it’s often a way to undercut people who present their opinions and arguments with “absolute certainty” that they’re right. The only problem is that the Dunning-Kruger effect itself is wrong.

Dr Danvers shows how the statistical methods used by Dunning and Kruger were flawed. When checked by others, the effects they supposedly measured did not differ from randomness. The upshot is that everybody tends to be a little overconfident at times—experts, ignoramuses and everybody in between. It’s a common trait in human nature. Replication tests using better methods showed that “Everyone was just a bit overconfident in their abilities, no matter what level they were at.”

—<Quote ends>—

Who knew that Dunning-Kruger itself was a failed study, just another demonstration of the replication crisis?

What a comeback! The very people claiming to be the most knowledgeable about Dunning-Kruger know the least about it.

How to Turn Dunning-Kruger Inside Out


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.