Different day, different diagnosis?

In medicine, the first step is an accurate diagnosis. Yet many conditions, including attention deficit hyperactivity disorder (ADHD), require physicians to rely on more subjective criteria such as observation of symptoms or behaviors. This opens the door for cognitive biases and external factors to influence medical assessments. In a recent paper published in the National Bureau of Economic Research, “Halloween, ADHD, and Subjectivity in Medical Diagnosis,” Harvard researchers spotted an opportunity for a natural experiment in a holiday that tends to make children hyper. Wondering whether changes in young patients’ behavior — related to the excitement of wearing a costume and collecting candy on Halloween — influence diagnosis of ADHD, they analyzed data on more than 100 million pediatric visits. They found a 14 percent increase in childhood ADHD diagnoses on Halloween compared with 10 surrounding weekdays.

We spoke with two of the study’s authors — Anupam B. Jena, the Joseph P. Newhouse Professor of Health Care Policy at Harvard Medical School, and Christopher Worsham, an assistant professor of medicine at HMS — about their findings. This conversation has been condensed and edited.

---

How is ADHD diagnosed?

Worsham: Speaking colloquially, ADHD is diagnosed by pediatricians and child mental health specialists over time. The goal is to assess a pattern of behavior across school life and life at home, gathering information from parents and from teachers. That said, the actual diagnosis is going to happen on a specific day. And so when the doctor is making that diagnosis, the conditions occurring on that specific day could influence whether a case that’s on the fence ends up being or not being [diagnosed as] ADHD.

Jena: There’s a formalism that goes into the diagnosis, but ultimately a diagnosis has to be made. It’s a snapshot of a given day. It’s not a laboratory test.

How does subjectivity come into play?

Worsham: The actual diagnostic criteria for ADHD have subjectivity baked in. Some of the diagnostic criteria are: “Is this kid fidgety? Is this kid moving a lot? Are they restless? Are they talking when they shouldn’t be talking?” If you are a kindergarten teacher and you’re looking at a kid’s behavior, you’re comparing it to your idea of how a kindergartner should behave. And that’s going to vary from teacher to teacher.

On Halloween, we get to focus on the subjectivity of the doctor. They’re going to have that one last piece, which is how that child is behaving in front of them on that day.

Jena: You might expect a response to Halloween in a child who either has ADHD or who is at risk of having ADHD because it’s an exciting holiday. Halloween is like a stress test.

Why does misdiagnosis matter?

Jena: Let’s say two or three children are being diagnosed. Two might be diagnosed on a non-Halloween day, and three on Halloween. That third child, that’s an extra medication prescription.

Worsham: Some of this goes to the broader discussion about ADHD. Approaching 10 percent of kids, more boys than girls are getting diagnosed. That’s a lot of kids. And the question is: Is this really a disorder? Is this some range of normal? Are medications the right answer? This is a huge debate in both medical and educational circles. There are concerns about both overdiagnosis and overtreatment of ADHD from kids who will not benefit and might be harmed by the drugs. And there are also concerns about under-diagnosis and undertreatment of kids in historically underserved populations.

Does your study shed light on ADHD itself?

Jena: What it highlights is the subjectivity of the diagnosis. Mental health diagnoses are different than physical health diagnoses. The government treats them differently. Health insurers treat them differently. The way they are diagnosed is different, in some sense necessarily. With mental health, there’s not a lab test that you can do. The question for us is how big of a deal is subjectivity and how do you measure what factors influence the subjectivity?

What controls do you think should be put in place?

Worsham: There’s a lot of literature supporting the idea of at least making people aware of the possibility of a cognitive bias. In this case, “Hey, it’s Halloween! You might be viewing this patient slightly differently than you otherwise would have. Remember that when you’re making a diagnosis.” At least planting that seed might help reduce some of the bias.

Jena: In my book “Random Acts of Medicine,” we talked about a finding a couple of years ago that showed that kids who are born in August are more likely to be diagnosed with ADHD than kids who were born in September. The reason is that kids who were born in August in states that have a Sept. 1 cutoff for entering school are the youngest kids in their class. And so their behavior is a little bit different. The implication is that those kids may not have ADHD. They’re just physiologically different because they’ve been alive for 20 percent less time than their peers who are in that same class who are nearly a year older.

If you are a physician who makes ADHD diagnoses, if you know that the child that you’re evaluating has an August birthday, you should at least pause, do a timeout, and say, “Am I making this diagnosis appropriately here?”

You can implement that kind of information very easily into an electronic health record. You have a child who’s being evaluated, and you are making a decision to write a prescription. There are multiple points at which the electronic health record could just flag you and say, “Are you aware of this possibility?”

Does this have any implications for other neurodevelopmental or psychiatric disorders?

Jena: Generally, it is interesting to think about situations where the doctor’s clinical decision could be affected by external factors that would make them more likely to make a diagnosis. For example, suppose a person is seen in a doctor’s office and the doctor is thinking about whether this person should be diagnosed with depression. Well, if a celebrity recently committed suicide or something was in the news about depression, maybe that doctor would make that diagnosis more often simply because it’s more salient in their mind.

Worsham: The other side of that coin is that this natural experiment is suited toward studying ADHD because Halloween brings out diagnostic criteria for ADHD in a way that it wouldn’t bring out diagnostic criteria for, say, an eating disorder.

The measurement is specific to ADHD, but it does tell us that maybe we should be looking for other circumstances because if we can show it here, there might be evidence of it under the right conditions for other diseases.

---

Research described in this story was supported by the Agency for Healthcare Research and Quality.