I was checking out of a hotel last week when I noticed a stack of customer survey cards on the front desk.
Someone–one of the clerks or perhaps even the hotel manager–had highlighted the positive end of the scale of the grid questions about service, room cleanliness, etc. The message was clearly, “Rate us as excellent!”
This isn’t the first time I have encountered such leading messages in “research.”
When I take my car to the dealership for service I routinely get a little speech before leaving that goes something like this: “We value you as a customer and want to provide the best possible service. In the next couple of days you’ll be receiving a phone call with a survey about your experience here today. We hope that you will rate us a 10 on all four of the questions they ask you, indicating we gave outstanding service.”
Now, I am all for customer service surveys as an ongoing indicator of how well a business or organization is performing. But I do have a problem with these kind of comments and indicators that can have an impact on how customers rate their experience. It is human nature to want to please others. (Well, some of us anyway). When people ask us for something, most of us want to give it to them if we possibly can.
So when a business or organization tells you how important a positive rating is to them, many people are going to help them out a bit. No, it won’t turn a filthy hotel room into an “outstanding” one just because it’s highlighted. But it just might make you rate them a 10 rather than an 8 or a 9. The overall, cumulative effect of this type of systemic bias makes the research results questionable, at best.
I used to work with a researcher who was absolutely scrupulous in crafting telephone surveys. Rather than ask a question such as “Did you find the information helpful?” he would add “or not.” So the question would read “Did you find the information helpful or not?” In this way, the respondent didn’t have to disagree with the question about whether the information was helpful. They had the option to answer either way.
Splitting hairs? I don’t think so. Looking at every question and how it is asked is the job of a thorough researcher. It’s also the job of the researcher to ensure that people who can influence the outcome are instructed not to give messages like “I love you. Do you love me?”