A while back I happened to be making dinner when the phone rang. It was a perfectly polite young woman conducting a research project for Amtrak. After confirming that I was, indeed, the same Robin Wedewer who regularly travels to New York City via the Northeast Regional train on Amtrak, she asked for a few minutes of my time to participate in the survey.
Of course, I disclosed that I am a marketing research consultant, which usually disqualifies me from participating in research of any type. But undeterred, this young woman said that didn’t matter at all. I agreed to participate.
As the call unfolded, it was evident that Amtrak was conducting a conjoint study to determine the different pricing options of the Northeast Regional compared with the Acela (the express train) and other travel options available to travelers. The interviewer marched me through a seemingly endless series of alternatives that went something like this:
You are traveling to New York City. If the Acela costs $70, the Northeast Regional costs $60, the bus is $45 and the plane costs $80, which would you choose?
You are traveling to New York City. If the Acela costs $80, the Northeast Regional costs $50, the bus is $30 and the plane costs $90, which would you choose?
Following each series of choices, my response was invariably the Northeast Regional. Regardless of the costs or options she proposed, I chose the Northeast Regional.
After about 20 minutes of this (Do you know how incredibly boring a 20 minute conjoint analysis interview is?), the woman told me those were all the questions and thanked me for my time.
Incredulous, I asked, “After 20 minutes my only response to your questions was the Northeast Regional. Doesn’t anyone want to know why I always choose the Northeast Regional?”
Her response? “Oh, the purpose of this study isn’t to understand why.”
Okay. I know that the purpose of the study was to determine the optimal pricing. That much was quite clear after 20 minutes. What I didn’t understand was why the researchers didn’t give participants—people who had essentially donated 20 minutes of their time to the cause—to tell Amtrak something they felt was important.
Even in a quantitative study such as this one, not everything of importance can be gauged through responses to closed-ended questions. Sometimes it is possible to glean something of glaring importance to a comment offered to an open-ended question.
That’s why whenever possible, I include a single open-ended question at the end of surveys to allow participants to volunteer their comments relevant to the topic of the survey. A typical question might be something such as: “Given the topics we’ve been covering in the course of this survey, is there anything else that you feel it is important for us to know or anything you wish to clarify about your responses?”
Often in the online format, people will skip the question altogether. But even more often people will take the opportunity to comment on an experience they have had with the association or company that is relevant to the research. Sometimes they even point out important decision criteria or other relevant information that wasn’t included in the survey.
In addition, asking for a final comment acknowledges that the participant is a living, thinking being that has made a contribution that should be honored by giving them the opportunity to speak.
Now, whenever I construct a questionnaire, I always think about that Amtrak survey. And I always make sure there is ample opportunity for the participant to say what they feel has been left unsaid. It’s been a valuable addition to many research projects.
–Robin Wedewer
P.S.
Why do I always choose the Northeast Regional over the Acela, bus or airplane?
Well, the bus is out of the question entirely. It takes longer to get to New York City from Washington, DC, by plane than by train because of the security and inevitable flight delays. And although the Acela is about 10 or 15 minutes faster than the Northeast Regional, it is always crowded at the station where I board, meaning I have had to walk two or three cars, dragging a heavy bag to find an open seat.
By taking business class on the Northeast Regional, I know there is a seat and in what car that seat is located—the first car of the train. There is no dragging heavy luggage through rail cars. The bonus? Business class is still cheaper than the Acela.
Follow Robin on LinkedIn