Last week, I wrote an essay about whether running a home-brew survey is the right decision relative to other ways of collecting data. This essay continues the series on running a survey internally by focusing on the one thing that dominates most conversations about survey construction. More than any other topic is a survey design team’s focus on survey length. While this focus is important, it is not as important as some would believe. Too often concerns over length will trump conversations about question topics and response categories. While it is important, it is too often the tail and not the dog. People are obsessed with survey length. I understand this. People don’t want to take a long survey and are afraid that their target audience won’t want to take one either. Heck, I hate long surveys as well, although, perhaps for reasons other than most. So, keeping your survey shorter versus longer is a good idea. Making survey length the most important element in your survey design, though is a BAD IDEA.
If you have decided to run a home-made survey and your primary concern is that your survey will feel burdensome or intrusive, perhaps you need to rethink surveying as your strategy. ALL surveys are burdensome and intrusive. No one in the history of the world has been sitting at home, alone, bored, and thought, “if only there was a series of questions I could answer to burn through ten minutes of my horrible life.” If you are concerned about being a burden, well, then, don’t survey your audience. Your primary goal needs to be a focus on making that burden worthwhile, both for you and the respondent. Length, in that frame, requires a more nuanced definition.
Length is about time and not question-count
Thinking about surveys as simply question counts misses how respondents view surveys. For most people, length isn’t a measure of distance; it is a measure of time. In my extensive background conducting and listening to others conduct telephone surveys, if a respondent requires about survey length, a vast majority will ask “How long will this take?” and not “How many questions is it?” You may think that these two things are inexorably linked, but they are not. If I give you a list of items and have you rate them all on the same scale, it will go faster than if I write each one out as its own question. Consider these two different ways to ask questions. The first:
And the Second:
- On a scale of 1 to 5, where 1 is Not Important and 5 is Most important, when thinking about Parking, how important is that to you?
- On a scale of 1 to 5, where 1 is Not Important and 5 is Most important, when thinking about Staff Courtesy, how important is that to you?
- On a scale of 1 to 5, where 1 is Not Important and 5 is Most important, when thinking about Nurses’ Knowledge, how important is that to you?
- On a scale of 1 to 5, where 1 is Not Important and 5 is Most important, when thinking about Ease of Scheduling, how important is that to you?
- Et cetera. I think you can see the pattern.
The first is easier to scan and process than the second. You would likely get all the elements in the first example rated in less time than it would take to get through the third question in the second example. In fact, you would likely find the second example very burdensome and would probably scan for the key differences in each question rather than reading them all verbatim. Further, the first example would likely be classified as ONE QUESTION while the second which would count as SEVEN QUESTIONS. From a respondent’s perspective, while both series cover the same topics, the first one feels shorter.
Length is about interest
If someone were to call me tomorrow and wanted to ask me a 50-question survey on music or the Chicago Cubs, I would be more than happy to participate. If they wanted to ask me 10-question survey about 17th century Wallachian1 architecture, I would likely demure. Interest implies a knowledge base, and both increase the respondent’s desire to participate. Since your survey involves an experience that they had recently, they are likely to have both knowledge and interest. You can extend their interest by making sure that the questions cover material that your audience would consider important. Assuring them that you will take the responses seriously as an effort to improve experiences is another way to feed that interest.
Length is about value and variety
The respondent has basic interest. You maximize that interest by making sure that the questions cover an array of different topics and avoid getting into the minutiae. It is a mistake to think that one with a passing interest will care as deeply as you do, so you want to sacrifice depth for breadth. For example, imagine you are creating a series of questions about an outpatient lab experience. One topic might be about convenience, especially as it relates to parking. It would not be hard to write multiple questions on parking.2 Further, if you only ask one question about parking and the respondent says that it was “below average,” your facilities management team will be upset that you did not ask more questions about parking so they would know what to fix. You will always have people who will want to dive deeper into weeds than a patient is willing to go. It is not that having five questions on parking and seven on the waiting room will make the survey long (which it will), but it will exhaust the respondent by forcing them to focus more on these elements much more than they ever have before. By the third question on parking, they likely stopped reading carefully as well. Nothing like a sense of boredom to make a survey feel long.
A ten-question survey that touches on parking, waiting, scheduling, non-clinical staff and clinicians will feel a lot shorter than a ten-question survey with five on parking and five on the waiting room. With the perceived diminishing value of the fourth question on parking combined with the obsessive focus on elements not seen as valuable as the friendliness and professionalism of the staff means creating a survey that feels frightfully long to the respondent. You want information, but you can only get it, if you keep the process feeling vital and varied.
Length is Pacing
I am not a cinephile, but I do enjoy watching movies. I have watched movies that only ran 88 minutes that still felt like FOREVER. I have watched 150-minute movies and have been riveted throughout. It is not about the length of the movie, but about the pacing. Movies that don’t waste your time with extraneous fluff can keep you immersed and not checking the clock. Surveys are the same way. There is a reason that your survey should ask questions in a thoughtful order. Most will start with the first thing you encounter and guide you through the entire experience. A survey that bounces around, asking questions in an odd order or about inconsequential stuff, feels longer because the respondent cannot form an opinion on, well, how long this will take. “You asked me questions about the nurses, and now we are back to questions about the nurses? Where are we going with this?”
This may seem obvious, but there are often two reasons why people skip around while asking questions. The first is because of the response scale. This will be discussed later, but for the moment, if your survey has questions that are 1-5 scales, agree-to-disagree scales, and perhaps a 0-10 scale, etc. you might decide to group questions with like scales together. This can be less confusing to the respondent, but you can end up revisiting the same topic with a slightly different questions with a different scale.
The other reason is borne from the concern that is trying to be addressed. Because people are so concerned about length, they will decide to put the most important questions first. That way, if the respondent bails mid-stream, they will at least get answers to the most important questions and can treat the partial-completion as a completed survey. By shuffling the questions around, though, you have disturbed the pacing of the survey and had people losing the thread. They cannot predict where you are going and will get frustrated with the perceived doubling-back. They are then MORE likely to stop before they finish.
Length is Process
How you engage your audience can affect how long a survey feels. For those who do not have a learning disability or a literacy issue, reading is faster than listening. For them, a written survey will always feel quicker than a telephone survey.3 Now, in healthcare, one cannot ignore the fact that there are those with disabilities or literacy concerns. So, this can influence your choice for survey mode, your decision to use a paper survey versus an electronic survey versus one run by an actual human asking the questions. So, while a written survey, or one that is executed on-line, may be less burdensome and feel less long, it may not always be the best option for your target audience.
If you are running your own survey, it is likely that you are running it either through a paper survey or an electronic survey hosted by an online survey vendor. Though as I type this, I am reminded of several surveys that are executed by a person, as part of a post-discharge conversation or by staff themselves, after a service, before discharge, or as a leader rounding conversation. Regardless of how you conduct your survey, there are ways to make sure your survey feels shorter.
For printed surveys, the visual is key. A mailed survey with multiple pages is less successful than a one-page survey, even if it is two-sided. This is both because it feels longer to have a two- or three-page survey, but also because there is always the possibility that one or more pages will fail to make it into the envelope that gets mailed back. For online surveys the focus needs to be on how the survey looks. You should make sure that the survey functions well, whether it is on a landscape laptop screen or a portrait smartphone screen. If the survey looks janky or the ‘next’ button is hidden, it is easy to just close the browser window or swipe out of the cellphone app.
Surveys conducted by people, either in-person or on the telephone, can be the most successful, but also have various challenges. Most are associated with the human conducting the survey. Do they sound happy or monotone? Friendly or distant? Conversational or formal? While each of these may not change the actual time it takes to complete a survey, they can have massive differences in how that experience feels.
In the end, length has more to do with creating an environment where a patient feels comfortable, even safe, to provide their perspective. You should consider length in your construction but be more aware of the softer side of length. Focus on asking meaningful questions that cut to the point. Use a mode that a respondent doesn’t have to fight with. If you do everything you can to reduce the barriers to participation, you have done far more to improve participation than by cutting your 11-question survey down to a 9-question survey.
1I originally selected “Romanian architecture” in the example but did a fact-check and realized that Romania wasn’t Romania in the 17th century.
2If you think it would not be easy to create multiple questions about parking, here are seven that I came up with off the top of my head, having been someone who has parked a car.
- Is there enough parking/Do you always find a spot?
- Are there enough handicapped parking?
- Is the parking lot in good condition? (No potholes, clearly painted lines, etc.)
- Do you feel safe walking from your car, relative to traffic flow?
- Do you feel safe walking from your car, relative to security?
- Does the signage in the parking garage clearly direct you to where you are headed?
- Have you used the valet parking option? What did you think of that?
3There are of course other issues with a written survey. Anyone who has looked at the HCAHPS mail-out survey will see that it is not necessarily intuitive, with its skip patterns and its double-column approach. But the extremes aside, a written survey feels faster.
PX Matters with Joseph Snipp 
Leave a comment