The limitations of surveys
Craig Cochran makes some excellent points in his article on the downsides of user surveys. In addition to his points about the timeliness of data and the difficulty in creating an unbiased survey (impossible, really), it's important to remember that a survey, like any research method, is good at some things and lousy at others. If you want data on real user behavior, go watch users in real life or via your log files instead of running a survey. If you want new ideas for solving problems or meeting unmet needs, asking users scaled questions probably won't help. But if you want to gather quantitative data on user goals and attitudes to generate persona segmentation, I find that surveys can provide helpful input. If you want to test a theory about what users think or what features they want, a survey is a fast way to gather helpful data.
