Do Demographics Matter? It Depends…

When designing a survey, if you write a question and the first response of the person taking the survey is, “It depends…” then you have written a crappy question.¹  But in a grad school philosophy class, the professor talked about how the only real understanding comes from wrestling with things that are at the same time both true and not true.² 

Patient experience is essentially balanced on this same line, between needing clarity but also needing to accept the grey area.  As I type this, I think this is the essential challenge with service especially in healthcare.  Staff demand clarity (Just tell me what to do) but complain about the lack of a clear cause-and-effect relationship (I did everything you said, and they still gave us a ‘7’).  We must be comfortable building clear action plans, but also comfortable with the fact that often, in the clinician’s eyes, this whole body of work can feel random.  More on that another time and in another essay.

Here, I will focus on one element of this debate—is who a patient is really important?  This may seem an odd question based upon the fact that this entire blog is called PX Matters, but the focus here is on whether a patient’s demographic data drives how they experience care and therefore needs to be factored into the process of providing them with remarkable experiences.  Let us explore this tension by looking at both sides.

No, It Doesn’t

I start with this side because this is where I started in my journey with patient experience.  If you have been reading my essays, by now, you know that I think patients are important.  But when I started this work, I was not convinced that it was valuable to burrow down the rabbit hole to understand the demographic makeup of a patient population in determining how to address the patient experience.  Instead, I found these conversations to be simply justifications for not trying to improve. 

• Best-practice is best-practice.  We know what patients want—they want communication, compassion, and the understanding that the care team is on the same page.  This doesn’t vary depending on who the patient is, or why they are in this hospital or clinic. 
• Action plans need to be simple.  Creating a PX process that starts with a decision-tree on the gender, age or ethnicity of a patient won’t get executed by frontline staff.  Successful action plans focus on ‘every time’ behaviors and not ‘every time for white women over the age of 50’ behaviors.  Requiring staff to consult a flowchart before talking to a patient is a PX non-starter.
• We are just making excuses for poor service.  Most conversations about demographic data seem focused on creating excuses for failing at service.  The overarching concern is that what I might do doesn’t matter because these people are just unreachable, unfathomable, irrational, or beyond hope.
  • First-time moms are nightmares
  • Hispanics don’t give top box responses
  • Oncology patients are just crabby
  • Young men in the emergency department are cocky and don’t listen
• Stereotypes are not useful.  A related conversation is just steeped in stereotypes.  I have often heard that when giving bad news, you should put a comforting hand on a woman’s or Hispanic’s arm, but you should never touch an Asian’s or man’s arm.  These things are ham-fisted at best and, frankly, offensive at worst.  Plus, there is no guidance if you are giving bad news to an Asian woman or a Hispanic man.
• Math doesn’t lie.  I once assembled a massive database with hundreds of thousands of patient surveys with patient demographics, community demographics, hospital demographics and unit demographics attached.  I wanted to see if any of these descriptives made it easier or harder to get desired responses.  Some day I will wander through that work here, but for the moment, I could find no statistical or useful impact of patient demographics on scores.  I would also do similar analysis on a specific hospital’s data and discover that, at best, gender or age might be the 12^th or 13^th most important variable in explaining variance.  This meant you should focus on all of the other eleven or twelve variables before you should start looking at patient demographics.

At base, focus on what you can control.  You cannot pick and choose your patients.  Your emergency department cannot go on diversion only for Hispanic men between the ages of 18 and 34.  Worrying about any demographic impact on experience is focusing on stuff that you cannot control at the expense of not doing the things that matter consistently.

Yes, It Does

It was clear, though, that a lot of the conversations about PX had patient demographics baked inside.  If you set aside the more objectively stereotypical comments, a lot of what drives the work is rooted in some basic assumptions about the kinds of patients we face.  For example, the assumptions we make about the different relationships obstetricians and gerontologists have with their patients. 

• It is the core of PX.  I often describe PX as “the rather outrageous notion that there is a human being attached to that broken leg.”  It would seem illogical to define it as a human interaction and experience and then remove the humanness from the calculation.
• Equality vs Equity.  Current language in healthcare focuses on the difference between equality (giving everyone the same thing) and equity (getting everyone to the same outcome.)  In other words, recognizing that different people need different accommodation.  This can include issues of physical disability, mental disability, or language issues.  Many cases involving alleged Americans with Disabilities violations that I dealt with evolved from treating all patients equally, but not equitably.  If you read patient comments, you will see that this difference drives many patient frustrations as well.³
• Generational and regional differences.  While this can swerve into stereotypes, different generations have different expectations in connectivity or immediacy in communication.  Some people want the ability to text their provider directly or get information via text or email.  There is certainly a technological divide in this country, which could be defined by age, rural/urban or along any socioeconomic lines.
• CMS says it matters.  The current CMS adjustments to HCAHPS data include weighting based upon patient-mix, mode and patient self-reported health and mental health status.  It does not include patient-specific demographic weights, but the implication in the weights they use is that the type of patient matters (med/surg/OB) and that a patient’s self-evaluation matters, which opens the door to broader indirect patient demographic impacts. 
• Math doesn’t lie.  While I could find no impact of patient demographics on scores, there is research that demonstrates that having patient/clinician concordance (similarity in race, gender, language, beliefs between patient and doctor) can lead to higher scores in patient satisfaction surveying.  So even if I could no evidence of a direct impact, there could be an indirect or collateral impact on scores.

At base, saying that patient demographics doesn’t matter fails the eye test.  While I am on record saying that the eyes lie, one can only see so much smoke or hear so much quacking before one assumes that somewhere there is a fire or a duck.

It Depends

Like so many bad survey questions or good philosophy questions, it really seems like the true answer is that it depends on context or positioning.  It is important to focus on consistent process and not get distracted by having several different variants that might add value but also can also overfit a model.  It is also important to realize that consistent execution does not lead to uniform results.  This does not mean the process is random, only that focusing on what we can control means that there can be externalities that can also impact outcomes on the margins. 

“It depends” also means that execution of consistent best practices is important, but it is also important to consider the reality of your patient population.  Increasingly appropriate healthcare cannot stop at the hospital’s front door.  Too often healthcare relies on patients behaving rationally in their own self-interest, even though we know that this is not a reasonable expectation of any humans, including ourselves.  A vast majority of a patient’s self-care and health management happens outside of our field of view.  We need to understand the ‘what’ and ‘why’ of this behavior, especially if it doesn’t align with our recommendations.  This concept is best captured in the conversations on social determinants of health; conversations on good health outcomes must include conversations about community safety, where patients can get access to healthy food, and safe households, where they can get supportive care free from abuse. 

This all reminds me of a time when I was working with a health system in California.  The health system, among other things, surveyed patients for their outpatient lab settings.  They had about ten labs scattered across their California footprint.  In looking at the data, almost all the labs had consistent scores across both the overall rating question as well as the other battery of questions.  One lab, though, had similar scores across the board, with one exception: a question about timeliness of care.  Their overall rating scores were also lower than the rest.  Statistical analysis revealed that this one lab’s lower scores were driven by the lower scores on timeliness.  Further, they were the only lab that had timeliness as a key driver to their overall performance.

In talking to the client, they were confused, as all the labs were run by the same set of leaders and used the same processes for executing both care and experience.  All labs let patients know about any timeliness concerns at check-in and let patients know if there were any new delays or issues, as they waited for their turn.  So all patients at all clinics were being treated equally.

I reviewed their data and noticed that the demographics for the clinics were very similar, but this particular clinic had four-times as many Hispanics taking the survey, as well as more women taking the survey than other labs did.  Our subsequent conversation led us to this realization.

This clinic was located on a bus line in a more urban setting.  Its patient population was predominantly non-white, lower-income workers with limited English proficiency.  This created two issues that combined to create this lab’s key issue.  First, the population was neither fully proficient in English nor consistently needing interpreters.  This meant that each lab encounter needed a few extra minutes to clearly discuss and complete.  Even though this might only be a few extra minutes with each patient, it meant that by midday, there might be a ten-minute delay in getting patients in.  Second, most of the patients arrived via public transport.  So, a patient with an appointment over their lunch hour might be significantly disadvantaged should that five- or ten-minute delay force them to miss their bus and be late getting back to work.

Did a patient’s demographic information matter to the scores?  Well, it depends on how you define that.  The data would NOT support saying that Hispanic women are crabbier than anyone else.  It would support saying that patients at this clinic put a higher importance on timeliness, but if you defined them as “impatient,” you are adding a value judgement that would likely color your response.  Telling the patients that they are being “irrational” or “inflexible” or “not understanding how appointments really work in a clinic,” likely just made the situation worse.

Perhaps the best way to define this is that patient demographics matter, but not in the way they are usually considered.  Strategies that are simply built upon these demographics are likely to fail.  However, strategies built upon how a patient population may interact with healthcare (which may have some origins in demographic descriptions) are more likely to succeed.  Done thoughtfully, this turns patient demographic information into something the organization CAN control.

¹Someday I will write about double-barreled questions and bad response categories.  For the moment, confusing a respondent with a poorly worded question will not yield serviceable data.

²This was not meant to be a brag, humble or otherwise.  That class kicked my butt and stole my lunch money every single week.  I was the only non-philosophy student in the room, and never in my life have I ever felt more out of place.  To this day, I can only imagine that I passed the class due to the kindness of the professor.  The class was on the Hegelian Dialectic and if you know what that is, you can understand why a statistics-oriented social scientist would want to take the class.  If you don’t know what it is, you can google it, though I am not sure I would recommend it.

³For those still not tracking this, consider that we may provide all patients with their test results through the patient portal, this does not mean that all patients possess the ability to access the internet let alone their patient portal.