This is how I felt this morning…
I dutifully – in a catch up mode after e-shipping off a set of client deliverables this mid-morning – completed several surveys sitting in my “Read Next” folder in my email system. Some were several weeks old. Some of the senders had sent a couple of reminders.
I didn’t think any of the surveys were very good – and when I complained to a peer – she commented about one that we shared in our email system – and shared the feedback that THE organization sending out this survey had received a couple of years back on this annual effort – from a long-time stakeholder – who knows what they are talking about…let me share…
The problem is with the words used in the scale. This typifies the problems that arise when questionnaire makers insist in making up scales, particularly Likert-Like scales rather than staying with convention and using the Likert Scale itself or a thoroughly researched Likert-Type scale. I suspect that if one ran a scale reliability test on these data, there would be problems.
I have been teaching a course in survey research methods for 20+ years and it continues to surprise me how people make up questionnaire items, accompanying scales, and so on totally ignoring the long established and well researched “rules of the road” and create mostly worthless questionnaires. Acting in ignorance, they get results and believe they are meaningful. It is akin to teaching school – we all went to school so we are experts on teaching. We all have seen and filled our scores of questionnaires, so of course we can make one up and use it. The items and scales make perfect sense to us, so they will to everyone.
As your discussions indicate, the scale in this study is not valid or reliable, and the questionnaire results are not really very useful.
Well, that was a complete waste of MY time – probably for all 3 surveys.
I dislike when THAT happens.
And the fact that this survey will lead to invalid results – that may or may not be acted upon – is more than just a buzz kill.
It diminishes the organization, and its employees/ members/ associates/ customers – and other stakeholders.
Hopefully – as in the past – we’ll all see evidence that this too shall have no effect – which I guess – is the good news.
# # #