Because we use a methodology superior to other needs assessments.
First, an important distinction. All needs assessments ask for information, some of which is objective, or factual, such as, "How many employees report to you?" and some is subjective, such as, "How well do you communicate with your employees?" The focus of our discussion will be on the methods of asking subjective questions.
As the word "subjective" implies, there is a judgment involved, which must first be made, then reported. The way a question is presented affects the judgment that will be made, which, in turn, determines the kind of data we will get. First, we will look at the most common approach to asking subjective questions.
Rating Scales — Absolute Questions
These are the questions we've all seen before. In fact, they are so ubiquitous, most people think there's only one way to ask questions about training needs. We're talking about the rating scale method. Here's what a rating scale looks like:
Q. How important are computer skills in your job?
Not Important
Very Important
Sometimes the individual choices will be labeled with words like "very much", or use adjectives like "low", "important", or "outstanding". Or you may see a statement, followed by "agree / disagree" choices, and maybe the choices are vertical instead of horizontal:
Whether the choices are labeled, or the scale presented vertically or horizontally, whether there are 3 choices or 10, the process is the same. The person completing the questionnaire has to make an absolute judgment about something, and check a box.
Here is a safe and easily defended conclusion about the use of rating scales in a "do it yourself" needs assessment you can get from an internet web sites: These surveys are not reliable, and, lacking reliability, they cannot claim to be delivering valid information about training needs. The reasons behind this conclusion are available for you in What's Wrong With Rating Scales?
Comparative Questions
Instead of asking for an absolute judgment, we can ask for a comparative judgment. A comparative question asks the respondent to make a judgment between two things rather than a judgement about one thing, like this:
Q. Which skill is more important in your job?
Dealing with ambiguity and change
Developing groups into effective teams
Here's another example:
Q. For which skill is a Supervisor more likely to be rewarded?
Making effective oral presentations
Dealing with ambiguity and change
Why Would I Want to Ask Questions This Way?
There's a practical answer and a scientific answer.
Practically Speaking...
The decisions we make in life for which we need good judgments are almost always comparative, rather than absolute. We have to make choices about how we will spend our time, allocate our resources, share our knowledge, choose right from wrong, choose right from right — these are all comparative choices. They aren't necessarily mutually exclusive, but they do involve priority decisions.
In the case of allocating training resources, all decisions are comparative, because almost all resources — space, financial, time, human — are limited, rather than unlimited. It is a constant process of deciding which skills and competencies are more critical to the strategy. We may think we are focusing on "how important" a skill is, when we are really looking at "which skill is more important — A or B?" Rarely do we focus on the absolute importance or value of a skill in making decisions.
Even when an organization uses an absolute, rating scale method to assess training needs, they don't look at the results for absolute information. They immediately start asking questions like "What are the salient training gaps?, what needs immediate attention?" They want to know the important training issues. They don't look at a single number and say, "Hmm, leadership is 63% — that's interesting." Any single single skill's score makes sense only when compared with the scores of other skills.
Scientifically...
People are better — exponentially better — at comparative judgments than absolute. As children, we learn to compare the relative weight of two objects in our hands. We know how to identify which of two persons is taller years before we learn about tape measures and average height.
In laboratory testing, individuals told to judge the length of a black line painted horizontally on the wall 10 feet away could rarely guess within 4 or 5 inches of the actual length of the line. But when presented with two lines, and asked to guess how much longer one line was than the other, they were regularly able to guess within 1/4 to 1/2 inch of the actual difference. The same studies have been repeated with different objects, distances, and sensory modalities with highly consistent findings. We are simply better at comparative judgments than absolute.
Conclusion
If your goal is to gather reliable data about training needs, employ a comparative survey approach rather than one that uses absolute, or rating scale questions. Comparative needs assessment tools are available here. We have been using them since before the internet, and we are experts in measurement theory. Contact us for more information on how we can help you.
