There are professionals whose expertise we rely on routinely and heavily in our lives: doctors, auto mechanics, crocodile catchers … In spite of our efforts to scour the internet for solutions to every conceivable life problem (including “how to lure a leprechaun”), we can’t be experts on everything. As such, it is customary and even comforting to hand over our problems to the relevant experts. For example, if the electrical system in my car causes problems, I will defer to an auto mechanic and, assuming the mechanic has been practicing for a while, I will likely trust his or her knowledge and experience. I probably won’t question the mechanic’s assessment of the situation, nor the method used to fix it, unless the mechanic tells me there’s a problem with my “dashboard-ma-dooly-wig”. In that case, I might find another mechanic. In most cases, however, I will rely on the expert’s judgment.
Unfortunately, in many areas of expertise, people (including experts) can be overconfident about their judgments and decisions and they may not recognize when they are wrong. When it comes to maintaining your vehicle, overconfidence of the mechanic may not be a major concern. If the mechanic misinterprets the problem, but is confident in his or her original assessment, your car will probably just make some weird noises, overheat, or stop working until it is reassessed and correctly fixed. When it comes to things like your health, on the other hand, overconfidence from the expert could have a permanent effect on your health. You might be misdiagnosed and, as a result, receive the wrong treatment: a potentially fatal situation.
Misdiagnosis is a problem growing in recognition. It is estimated that 10-15% of cases are misdiagnosed. Prominent researchers assert that this misdiagnosis may partially stem from physician overconfidence. Can physicians, who have attained one of the highest levels of education, be overconfident in their area of expertise like mechanics or other experts can be? Until now, the presence of physician overconfidence had not been directly assessed.
In a study that came out Monday by me and my colleagues; Hardeep Singh, Velma L. Payne, Derek W. Meeks, and Radha Rao (found here); we assessed the relationship between physicians’ confidence and their accuracy in diagnosing patients. We mainly wanted to know whether physicians’ confidence and accuracy are related or if physicians show overconfidence. We also wanted to know how confidence and accuracy related to the use of additional diagnostic resources (such as lab tests or consultations).
In the study, 118 physicians from all over the U.S. were given 2 easier cases and 2 more difficulty cases to diagnose (they were not labeled as such to the physician participants and were presented in a random order). These cases were presented online in a web-based format, but were based on real patient cases (we’re not making this stuff up!). We presented the cases in an evolving manner, such that the physician participants first saw the symptoms the patients came to see the doctor about, along with the physical examination findings. Then, the doctors got more and more information as the cases unfolded. Additional information included general and more specific diagnostic tests (like blood tests and imaging). In total, there were four chunks of information that the physicians got for each case. At the end of each chunk, we asked the physicians to tell us what diagnoses they were considering and we also asked them to tell us their confidence in each diagnosis they provided (on a scale of 0-10 with 10 being the highest level of confidence). We also asked physicians what additional things they would do to solve each case, including asking a colleague for help, looking up reference materials, or ordering additional diagnostic tests.
For the two easier cases, the physicians got 55% of the cases correct: for the more difficult cases, they only got 5% correct. It was not surprising to see a difference in accuracy between easier and more difficult cases (note- we purposely chose pretty difficult cases overall, so don’t worry about the overall performance of the physicians!). What was surprising was that confidence barely budged as physicians went from easier to more difficult cases. On a scale of 0-10, physicians rated their confidence as 7.2 on average for easier cases, but as 6.4 on average for more difficult cases. So, physicians still had a very high level of confidence even though their accuracy dropped to a mere 5%! We also found that when physicians were wrong, their use of additional resources did not increase, meaning they were not getting the additional help they needed to solve the cases. We did find that they asked for more diagnostic tests when their confidence was lower, but unfortunately this lowered confidence was not related to lower accuracy. So, even though they would order additional diagnostic tests, they wouldn’t necessarily do it at the right times (when accuracy was low or when difficulty was high).
Taken altogether, this study shows that physicians, like other experts, can be overconfident when they are wrong or when they are solving difficult cases. This overconfidence can inhibit them from choosing additional diagnostic resources further decreasing their potential to accurately diagnose patients.
This is a sobering finding. Physicians, who have been thoroughly trained in their fields and who are arguably some of the smartest people around can be overconfident and can misdiagnose you. So, what do we do with that information? Well, in relation to medicine, research in this particular area is just beginning, but a few things have been suggested. Communicating with your doctor accurately and reporting all of your symptoms and history (even if you think things might not be related) might help. Research has shown that one of the most error-prone parts of the diagnostic process occurs while patients are being examined by their providers and are being asked about their medical history (usually the error is related to insufficient information gathering). If the doctors aren’t truly seeing the whole picture, this might lead to overconfidence and eventually to misdiagnosis. It has also been suggested that people should get second-opinions. This might help combat overconfidence in any area of expertise- not just in medicine. So, the next time you get a diagnosis from a doctor or the next time you hire a crocodile catcher, just know- they might be overconfident and they might not perform as well as they think they will. This doesn’t mean you should ignore their advice or mistrust them- it just means we should be open with them and provide them feedback if our situation isn’t improving or if we think our problem might be something else.