One of the bigger frustrations in working in the patient experience space is the preamble you often must cover with your clinical audience before getting to the topic you want to talk about. This is the justification as to why this audience should care about the patient experience enough to pay attention to whatever initiative you are encouraging them to adopt. I used to think that this was because clinicians believed that this was all about warm fuzzies based upon data that was drawn from a social science methodology using sampling and attempting to measure feelings. This infused the conversation with a relativity that the clinicians just could not get past. These questions asking about perceptions and opinions were just too imprecise to accept and the behaviors being suggested to address gaps were also too imprecise to be universally successful.
This meant that when I took a job for a hospital system as their clinical data analytics director, I thought that I had found the cheat-code that would allow a numbers nerd such as myself to talk about how behaviors affected outcomes and have the clinical audience accept this as the truth. After all, this was clinical data. Hard data. Mortality data. Cause and effect data. This youthful enthusiasm lasted through exactly one meeting.
This was when I learned that clinicians did not dismiss PX data because it was too touchy-feely. They dismissed it because it did not conform to their previously-held beliefs. Likewise, they dismissed my hard analysis on outcomes because it did not align with their perception of reality. I learned an odd but important distinction in healthcare. Healthcare is a science-driven industry, but it is not a data-driven industry.
The relationship between healthcare and data is complex and nuanced, so me saying that they don’t use a data-driven methodology is simplistic. It is perhaps more accurate that they use data in retail one-on-one encounters with individual patients, but not in wholesale decision-making about broad categories of diagnoses. If lab results show that my cholesterol is high, it is true that this will take us down a clear corrective path. But using data predictively is something that I have found hospitals are not keen to engage with. For example, I discovered that the number one reason why patients were not deemed compliant with optimal diabetes testing was the absence of a second a1c test in a calendar year. I also discovered that over a third of these patients needing a second a1c test had seen a primary care specialist for a non-diabetes-related need in that calendar year. I realized that if the system been able to harvest a second a1c test out of just half of these encounters, the system would have gone from failing to meet a standardized goal on the measure to dominating the measure. Now, yes, there are challenges here. The biggest hurdle is to remind a doctor or nurse to ask. It seemed like the lowest of low-hanging fruit. And yet, by mentioning this story at this time, I assume that you, the reader, can guess how this turned out. There was significant antipathy to institute any procedures to close this very simple gap. The data was clear on how to successfully meet a measure (oh, and also help patients avoid complications associated with their diabetes) and yet, there were more questions and concerns over the finding than willingness to accept it. The same audience that was science-driven to accept an established a1c goal of under 7% were resistant to a data-driven goal to help patients stay on the path to diabetes maintenance.
It is difficult to close this gap, even when we know it exists. There is well-documented evidence that there is gender bias in some serious diagnoses. For example, doctors are more likely to miss a case of congestive heart failure (CHF) in women as opposed to men. While there are symptomatic reasons for this—CHF in women often presents differently than in men—since this has been well-understood for over a half-century, it is appalling that this gap still exists. The same is true with CHF in non-whites. The gap exists even when controlling for social determinants of health and access to consistent care. I present this simply to illustrate that even when there is clear data to support a concerning trend, healthcare does not use data to drive wholesale decision-making which can, in turn, impact retail care management.
This is not about kicking healthcare in the shins for being slow to move, though I suspect this essay could be read this way. It is simply examining why healthcare, ever eager to embrace science, is so slow to embrace data. This trope seems firmly baked into the general perception of healthcare. I cannot think of a single book, movie, or television show that doesn’t subscribe to one, and often both, of the following ideas:
- All care involves the unique, rare, and mysterious affliction. Even if the reason why the patient arrives is mundane or standard, like chest or abdominal pain, the cause is always some bizarre thing that no one has ever seen before.1 It defies logic and all test results are confusing or conflicting.
- This weird disease or issue would certainly be missed, but for some wizened, usually irascible, doctor who no one believes, making him right when everyone else is either wrong or simply baffled.
These stories downplay the value of data in favor of gut instincts. They often suggest that if the team had simply followed the data, they would have gone down the wrong path. The only saving grace was that lone voice, playing by their own set of rules and saving the day.
I am not sure if art is simply imitating life or if we have arrived at a moment where it is a feedback loop, where the media image is now influencing contemporary care provider self-image. Further, the training clinicians receive tends to stress the costs of being wrong and therefore puts a high premium on your confidence and your ability to make the right decisions. While I appreciate the desire to bring gravity to the profession, it also puts great weight on trusting your knowledge and instincts in the face of opposition. Again, this is understandable, but if not balanced with humility and feedback from other professionals, this becomes as helpful as any other bias that people possess. This preconception weighs personal experience over broad patterns in the data. When one factors in the ‘terminal uniqueness’ that healthcare suffers from, it can be difficult to move someone off of a predisposition. Sometimes this is annoying and sometimes this can be life-threatening.
I am immediately reminded of the work I have done on sepsis mortality.2 One of the biggest challenges hospitals have faced is the management of sepsis. It is pernicious, deceptive, and when it appears with a host of comorbidities difficult to beat. One of the first tasks I had in working with the clinical data was to identify where the largest opportunities we had as a system to improve outcomes while managing costs. I created a bubble-graph3 that graphed performance against expected length of stay, expected mortality, and number of cases. When graphed, the usual suspects like acute myocardial infarction, CHF, diabetes complications, pneumonia, COPD, and others were tightly crowded. Each one had opportunities for marginal reductions in mortality and/or length of stay. The lone outlier, in length of stay, mortality and total case count was sepsis.
In looking at sepsis, I realized through chart review that our sepsis mortality was about 20% lower when we quickly instituted the “sepsis six” interventions. I calculated the length-of-stay and mortality data by individual physician and overlaid this data with the use of the sepsis six bundle. While I could not prove causality, I demonstrated that there was a correlation between an individual doctor’s lack of using the best practice interventions and their mortality rate. I then also looked at the mortality rate against the most-often missed interventions. I learned that the two interventions most often correlated with mortality were the failure to draw blood cultures to identify the specific organism before administering antibiotics and the failure to initiate aggressive IV fluid resuscitation.
Now, I know that some of you have dozed off there, not expecting a deep clinical dive. The point of my story is that the analysis provided a specific area of focus, the specific doctors who needed education, and the specific education that they needed. I naively thought that this was all I needed to do. I presented the data to the medical leadership and handed it over to the clinical informatics and skills training departments for execution. I returned to my office to await a call from the Nobel Committee4 and my bonus check for saving the organization millions of dollars every year.
In reality, nothing changed. In the best of all worlds, change is hard. But in this case, the lack of change could be traced back to two unsurprising and totally foreseeable issues.
- The doctors in question came up with specific reasons for why they did not execute the best-practice bundle in each case. Indeed, they often argued that each case was unique and therefore could not be grouped in this very broad analysis.
- The folks in the quality department pushed back at the analysis, citing other reasons for the mortality issue. These included age and gender of patients as well as the fact that most of the sepsis deaths were patients who were transferred to the flagship hospital, so they would die on their books rather than another hospital’s tally. In other words, NOT THEIR FAULT.
Now the doctors are right for the same reason that they are wrong. In statistics there is the concept of the ecological fallacy. This is the idea that just because you can identify broad patterns in data this does not mean that each specific case is attributable to that broad pattern. For example, as earlier stated, women are less likely to have their CHF correctly diagnosed. This does NOT mean, though, that any individual woman who was misdiagnosed was misdiagnosed because they were a woman. I said that patients who did not receive these specific interventions were more likely to die. This does not mean that the patients called out died because they did not receive these specific interventions.
I would be wrong in saying that Patient Mary died because Doctor X did not follow appropriate protocols. But by denying the Mary died because she did not get aggressive fluid resuscitation still ignores that Mary’s chance of survival was diminished because she did not get fluid. Mary’s life or death is predicated upon a number of different decisions made by a bunch of different people, some beneficial and some harmful. They all get combined in a big logistic regression of life and death. I would say more, but as my friend Adam has said, the first rule of logistic regression is that you don’t talk about logistic regression. So, my response to the doctors was not about placing blame about past decisions, but on focusing on consistent steps to improve the odds of the next patient.
The concern expressed by the local quality department was easier and harder to deal with. A simple review of the origin stories of all of these sepsis patients easily illustrated that sepsis patients that arrived by transfer had lowest mortality rate of any sepsis patients in the hospital.5 The patients most likely to die were those that came through the hospital’s own emergency department. While age and comorbidities were a factor, they were less a factor than the execution of the standard best-practices. This seems straightforward, but the challenge is that the erroneous perception of the problem has had a multi-year head start over the facts. I was surprised, then annoyed, then resigned to the fact that I would have to correct people for years afterward when they parroted the myth over the facts. Whether this was comfort of old habits, the difficulty in overwriting bad data in the brain, or the disquiet of realizing that the problem was staring at you from the mirror all along, these falsehoods are pernicious.
In the end, my desire is to not throw shade on hard-working and well-meaning clinicians. It is simply to call out a blind spot that healthcare sometimes has. The same thing that can make clinicians so good in one-on-one interactions can lead them astray in the big picture. While it would be satisfying to cast myself as the wizened and irascible statistician who is right when everyone else is wrong, I also have blind spots which means I also have to be wary. Just because a model is effective in explaining past data does not mean its implementation will work indefinitely or won’t inject new gaps later on. But that is the subject of another essay at another time.
1Unless the chest pain is nothing. But here, this is mostly used to drive personal change in a character. It wasn’t a heart attack, but this is the wake-up call to make serious changes in their lives. We almost never find out if they were successful.
2I am sure I have mentioned this before, but good stories are good stories for a reason.
3For those that don’t know, a bubble graph is a graph that essentially can show three dimensions in a two-dimensional space, where the x-axis can be performance against expected length of stay, the y-axis is performance against expected mortality, and the size of the bubble is sheer volume of cases.
4There is no Nobel Prize for statistics. Another example of how the man is keeping the honest hard-working number nerds down.
5Hindsight probably makes this easy to understand. These are generally patients who have had their sepsis properly diagnosed and have started receiving appropriate care. They are transferred because the flagship hospital may have technology and special staff needed for continuing that care. These resources have been notified that this patient is coming and so they are ready to start immediately upon arrival.
PX Matters with Joseph Snipp 
Leave a comment