September 12, 2006

The limitations of surveys

Craig Cochran makes some excellent points in his article on the downsides of user surveys. In addition to his points about the timeliness of data and the difficulty in creating an unbiased survey (impossible, really), it's important to remember that a survey, like any research method, is good at some things and lousy at others. If you want data on real user behavior, go watch users in real life or via your log files instead of running a survey. If you want new ideas for solving problems or meeting unmet needs, asking users scaled questions probably won't help. But if you want to gather quantitative data on user goals and attitudes to generate persona segmentation, I find that surveys can provide helpful input. If you want to test a theory about what users think or what features they want, a survey is a fast way to gather helpful data.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)