Thursday, October 22, 2009

Screening Phase

Examine data for five different kinds of possible errors:
  1. Lack of data – Do some questions have far fewer answers than surrounding questions?
  2. Excess of data – Are there duplicate responses?
  3. Outliers/inconsistencies – Are there values that are so far beyond the typical that they seem potentially erroneous?
  4. Strange patterns – Are there patterns that imply cheating rather than honest answers?  For instance, does a respondent alternate between ratings of 4 and 5 on every other topic in a matrix question?
  5. Suspect analysis results – Do the answers to some questions seem counterintuitive or extremely unlikely?