Questionnaire Validity

The validity of a questionnaire relies first and foremost on reliability.  If the questionnaire cannot be shown to be reliable, there is no discussion of validity.

But even reliable instruments may not be valid if they are employed for situations they were not designed for.  A good example of a questionnaire that may have high reliability, but poor validity is a standardized questionnaire that is used over and over in hundreds of companies.  These instruments are marketed aggressively using promises of "industry norms" to compare your results with.  Validity is not a characteristic of a particular instrument, attached to it in a way that ensures it will always produce accurate information no matter where or when it is used.  If you want validity, you have to be able to demonstrate validity in your situation; it is not built into the instrument.

But there is good news.  Demonstrating validity is relatively straightforward, compared to reliability.  If you have reached this point and have a reliable instrument for measuring the issues or phenomena you are after, demonstrating its validity will not be difficult.

How Do We Measure Validity?

While there are detailed and technical ways of establishing validity that are beyond the level of this discussion, the following are brief descriptions of the three basic approaches.  All proofs of validity employ one or more of these methods:

• Content Validity – If the content of a test or instrument matches an actual job or situation that is being studied, then the test has content validity.  For example, a Training Needs Assessment for middle managers should have content (such as skills, activities and abilities) relevant to the jobs of middle managers.  Skills that pertain to landscaping workers would not be appropriate in a needs assessment instrument for managers.
• Predictive Validity – This form of validity comes from an instrument’s ability to predict an outcome or event in the future.  If a questionnaire or instrument is developed to assess “promotion potential” of a group of newly hired workers, the results of the test should be able to predict which of the group will actually be promoted.  The predictive validity of the instrument is shown in the correlation between the scores from the test and the persons promoted.
• Construct Validity – This form of validity derives from the correlation between the test or questionnaire and another instrument or process that measures the same construct.  The Myers-Briggs Type Indicator (MBTI) is a well-established test of personality types.  A new instrument developed to assess the same characteristics would have construct validity if the scores from the new instrument correlated highly with the scores from the MBTI.

These are the methods for proving validity.  Providers of questionnaires and surveys who are unable or unwilling to talk about validity in those terms (content, predictive, or construct) should be avoided.  You will sometimes hear discussions of “face validity”.  Despite the use of the term, face validity is not a form of validity assessment.  It is simply a subjective appraisal of how an instrument appears to a person who examines it.  There are numerous examples of valid instruments without face validity and completely bogus instruments with loads of face validity.

ReliableSurveys.com has developed hundreds of reliable and valid instruments for organizational research.  We would welcome the opportunity to discuss your situation and hope you will contact us.