Conducting surveys is a widely used method of data-collection, especially when one is trying to use the opinion of people regarding a product or service. The biggest advantage of this technique is the direct hold of "Voice of Customer". It also proves to be a useful tool in those situations where the data actually doesn't exist is very difficult to get hold of, like salary surveys, college rankings etc. In such cases, certain organizations conduct a survey of professionals, or colleges to gather the data, analyze it, quantify the results, and make interpretations.
This is where the problem occurs, the interpretation, and not the data. Recently I read the IndustryWeek's 2009 Salary Survey. Conducted and published annually, this survey targets the professionals in the manufacturing industry. This survey is quiet detailed and presents the average salary results by industry, education level, geographic region, race, company-size, and what not. You can see the detailed Charts & Tables here. But writing about this survey is not my point, however the interpretation is.
The summary of the survey, as expected due to the current economic scenario, is:
'When asked, "How satisfied are you with your current job?" 76% say they are "very satisfied" or "satisfied," which is actually a slight bump up from the 74% response rate in both 2007 and 2008. And when asked about their choice of manufacturing as a career path, 80% say they are "very satisfied" or "satisfied," a slight dip from the 83% in 2008 but slightly better than the 79% in 2007. Clearly, both resiliency and pride are still alive and well in the U.S. manufacturing industry."
Now this is a contradiction, isn't it? How come the people are satisfied even when the salaries are going down, and rather worse, even jobs are going away by thousands? The answer lies in the paragraph immediately following the above, which, in my opinion, is the biggest pitfall of "Survey & Interpretation" technique.
"In this year's survey, nearly 1,700 readers participated in our anonymous survey, ........"
Now guess, who are the regular readers of IndustryWeek??? The professionals in manufacturing world...obviously. So the survey itself is limited to only those who are somehow related to manufacturing profession. no wonder, so many of them are "satisfied" or "very satisfied". Probably the people who are not satisfied have already changed their profession or didn't bother to respond to the survey.
Isn't it similar to Apple sending a survey to all iPhone users asking how much they like their iPhone? Probably the people who don't like it have already switched to some other phone.
Thus, one needs to be extremely careful in selecting the sample from which (s)he wants to draw his survey results. Your objective is to get the best out of this tool and use the results to improve your products or services. And be aware of your sampling constraints while drafting the survey questions.
This is where the problem occurs, the interpretation, and not the data. Recently I read the IndustryWeek's 2009 Salary Survey. Conducted and published annually, this survey targets the professionals in the manufacturing industry. This survey is quiet detailed and presents the average salary results by industry, education level, geographic region, race, company-size, and what not. You can see the detailed Charts & Tables here. But writing about this survey is not my point, however the interpretation is.
The summary of the survey, as expected due to the current economic scenario, is:
"As the U.S. economy gets leaner and meaner, IndustryWeek's 2009 Salary Survey reveals that the average salary for manufacturing management has dropped to $95,248."
But what prompted me to think about the survey methodology is this:'When asked, "How satisfied are you with your current job?" 76% say they are "very satisfied" or "satisfied," which is actually a slight bump up from the 74% response rate in both 2007 and 2008. And when asked about their choice of manufacturing as a career path, 80% say they are "very satisfied" or "satisfied," a slight dip from the 83% in 2008 but slightly better than the 79% in 2007. Clearly, both resiliency and pride are still alive and well in the U.S. manufacturing industry."
Now this is a contradiction, isn't it? How come the people are satisfied even when the salaries are going down, and rather worse, even jobs are going away by thousands? The answer lies in the paragraph immediately following the above, which, in my opinion, is the biggest pitfall of "Survey & Interpretation" technique.
"In this year's survey, nearly 1,700 readers participated in our anonymous survey, ........"
Now guess, who are the regular readers of IndustryWeek??? The professionals in manufacturing world...obviously. So the survey itself is limited to only those who are somehow related to manufacturing profession. no wonder, so many of them are "satisfied" or "very satisfied". Probably the people who are not satisfied have already changed their profession or didn't bother to respond to the survey.
Isn't it similar to Apple sending a survey to all iPhone users asking how much they like their iPhone? Probably the people who don't like it have already switched to some other phone.
Thus, one needs to be extremely careful in selecting the sample from which (s)he wants to draw his survey results. Your objective is to get the best out of this tool and use the results to improve your products or services. And be aware of your sampling constraints while drafting the survey questions.
0 comments:
Post a Comment