Few will argue with the idea that understanding how patients feel about their experiences is important. The conflict over executing a survey comes from the logistics of the work and the fear of doing the work poorly. This is why there are plenty of vendors out there who will gladly exchange cash for the effort of surveying your patients. All the ones I have worked with or worked for do a very good job of this. But there are still several audiences for whom a contracted vendor is not a viable option. I previously mentioned a Skilled Nursing Facility that wanted to survey their patients, but the quoted price from a vendor was greater than their yearly profit margin.1 When the survey process itself was the difference between operating in-the-black and in-the-red, that is a difficult pill to swallow. This is how people come to the decision to run their own internal surveys. I often call these “home brew” surveys because just like home-brew beer, these surveys are created by people whose primary job is not this work, and the product can vary from reasonably good to downright abysmal. This essay will start a series on important things to consider, if you feel like running your own survey is the best way to gather feedback, starting with whether this is really the best model for you.
Before even setting down the road of running an internal survey, it is important to understand that this is simply one way to get feedback from your audience. There are others. You could round on your patients or staff to get feedback. You could run a focus group with patients, either as a one-off or as part of a Patient and Family Advisory Council. There may be existing data sources, or perhaps you could simply add a question on an existing survey. There are advantages and disadvantages to all these approaches, just as there are advantages and disadvantages to creating a survey. Let us start by looking at some of those advantages and disadvantages.
Advantages
The biggest advantage of surveying is in efficiency of data collection. I will often talk about retail versus wholesale approaches to elements in patient experience. I define retail as individual and specific. It is tailored to a specific person and their experiences but is also very time-intensive to collect. I define wholesale as broad and general. It foregoes specific details and limits the space for a person to express themselves in order to create a system that will allow for more data to be collected more quickly. A survey can have significant start-up costs but will allow for a very cheap way to get data, albeit data that is one-size-fits-all. You are essentially defining the most important topics or issues because they are the ones you have questions for, while you are minimizing other issues or topics, because you ignore them in the question list. Asking a “Is there anything else you want to add?” verbatim question at the end is thin gruel in terms of allowing a patient to define the experience for themselves. But if you want a lot of standardized data, the efficiency of a survey is hard to beat.
There are some topics or situations where a patient is more likely to be completely honest than in others. Sometimes the anonymity of a survey can provide people the space to be honest. If you survey me about you and your team, I might tend to provide overly positive responses because I want to be nice or I want to avoid conflict or “getting someone in trouble.” This is often called “acquiescence bias.” Further, sometimes it is easier to agree with someone than it is to devote mental energy to disagree, since we know, this will lead to conflict and the need to support one’s position with logical thought. Further, if a patient is in the middle of their experience (you round on them on Day Two of a Three-Day hospital stay), they may be less than honest for fear of retribution, for example, if they say a staff member was rude. Moreover, there are some topics that can feel sensitive or private, and talking to a person may have a patient be less than forthcoming. Subjects like sexual problems, domestic violence or even incontinence can have patients feeling uncomfortable giving feedback to someone they are looking in the eye. Giving people the space to anonymously provide feedback is thought to provide a more honest appraisal.2
Disadvantages
I wrote an essay on how surveying for every element of a hospital stay can be counterproductive, since it builds a transactional relationship into each of those elements. It essentially breaks down each encounter into its own self-contained universe, rather than treating it as part of a larger gestalt. Hospitals pay a significant amount of lip-service to breaking down silos, so forcing every element into its own survey bucket runs counter to this claimed desire. This can have a collateral impact on staff. Normally, we want to encourage staff to be collaborative and cover us, if we stumble. But creating a survey can provide staff with the excuse not to engage. This is not meant in a negative or petulant way, but simply that as a nurse, I might be less inclined in my busy day to inquire about breakfast, since I know that there is a QR code on the menu. If you want staff to integrate—where a housekeeper can get a blanket, or a nurse can address gaps in food service—don’t provide them with an excuse to avoid the interaction.
Not all surveys are equally important. With all due respect to lab, food service, ambulance service or dialysis (all places where I have seen home brew surveys used), your surveys may be doing more harm than good. The concept of “over-surveying” is something that people can immediately relate to, since we all feel like we cannot cross the street without someone asking us about the experience with the traffic light. Aside from the general annoyance, though, this can lead to a bigger problem. People will often take the first survey they get and forego taking any additional follow-up survey leading to a drop in participation for the more important surveys. This is because of broad frustration with the request (how many of these things do I need to fill out?!?) or because of an honest belief that they took the survey already. For example. while post-discharge phone calls (PDPC) are a powerful tool to improve compliance with care and improve overall satisfaction with a hospital stay, if the PDPC asks a lot of rating questions in addition to questions about a patient’s understanding of their discharge instructions, it can depress participation in the HCAHPS survey, since the patient will think that they already answered these questions.
The reality of brewing beer at home is that it will almost certainly cost more than buying a premium six-pack at the store. The average home-brewer is not doing this to save money, but for the pleasure and challenge of the process. Anyone who starts home-brewing, though, should be aware that there are costs here. The goal must be to create something you cannot get from a vendor and not the goal that it will necessarily be cheaper. Likewise, someone creating a self-executed survey is not likely to save much money. The creation of the survey, targeting respondents and managing the data flow back and forth (all subjects of other essays), takes time and effort. People often forget that just because you are not cutting a check to a vendor, you are still spending money. Look around the table where you are working on this survey and add up the salaries of the people in the room for the hours needed to create it. Consider the hours a week or month required to do data entry, data summary, data visualization and data reporting. Don’t forget about the time to build action plans or in some way address the issues the survey discovers. Factor into this that it will take longer than you think to do these things, since the people doing the work are not necessarily expert at the work. You may get lucky and have dedicated and skilled resources in your organization. The irony, though, is that the more likely you have access to these resources, the more likely you will have to account for their time and officially document time and requests for those resources.
It may be more difficult to reach your target population than you think. If you want to target recipients of your service, like people who got food service while an inpatient or people who got an outpatient lab draw, that is straightforward. Often, though, the desire is to target people who are underutilizing your service. Consider food and nutrition services wanting to understand why a patient didn’t eat their breakfast. Or, why the patient only ordered two meals in the day and not three. These folks are more difficult to reach than one might think. After all, if I DON’T order lunch, am I likely to take a survey as to why? If a tray comes back to the kitchen untouched, how do we get that person to answer questions as to why they didn’t eat. And that assumes you know what room sent back a full tray.
One statistic that is popular to track with scheduling is the number of phone calls that come in but are abandoned before the scheduler can pick it up. Whether it is because the line continues to ring or because the person feels like they were placed on permanent hold, people hang up. Understanding why they abandon their call may feel obvious as it simply took longer for someone to get to them than the person felt willing to wait. But the follow-up questions of
- Did they call back? If so, how many times? If so, did they finally get what they needed?
- When did they call back? In five minutes, five hours or five days? Did this delay access to care?
- Did they not get the care they needed? Did they get the care from an independent provider?
- Did they try another form of contact, like email or an on-line scheduling portal?
It is highly unlikely that you will get answers to these questions because you don’t know who these people are. And if you did, they are unlikely to want to help YOU since you could not help THEM.
By only surveying those who keep coming back, we can learn what we need to do to keep those people happy. And that may incrementally grow our market share. But, in healthcare, there can be very important reasons why someone doesn’t choose you—from the availability of needed specialists, to changes in insurance coverage, to a change in office hours, to a change in services provided in some locations. None of these things will pop up in a survey of users. All these things will limit your ability to convert non-users to being active patients.
Now you may wonder why someone who has spent a large amount of his life dedicated to surveying patients would be so critical of surveying. I am not. I think it is a fantastic tool. I also know that it is misused and poorly executed and can be a colossal waste of time, money and trees. It should not be treated as the answer to all of one’s data questions. Surveying is a hammer and not all needs are nails. If you have decided that your need is a nail, we can discuss how to design the right hammer.
1A quick note for those who got their feathers ruffled at the notion of “profit” associated with healthcare. I don’t want to wind down a rabbit hole, but briefly, capital improvements on the facility, the ability to expand services, the ability to provide assistance to patients who need care, but cannot pay for it, the ability to create any ‘rainy day’ fund all presupposes an organization is not spending more in daily operations than it is taking in. I will save my opinions on the relative costs and benefits of any country’s medical-industrial complex for another essay, tentatively scheduled for publication on the 35th of Neveruary…
2Attentive readers know that I have stated that anonymity is not always what a patient thinks it is. That is, again, a subject for another essay, but for the moment, when I am talking about anonymity here, I am focused on a patient’s perceived anonymousness and not the reality of that perception.
PX Matters with Joseph Snipp 
Leave a comment