Abstract:

Multiple choice tests that are scored by formula scoring typically include instructions that discourage guessing. In this paper we look at test taking from the normative and descriptive perspectives of judgment and decision theory. We show that for a rational test taker, whose goal is the maximization of expected score, answering is either superior or equivalent to omitting – a fact which follows from the scoring formula. For test takers who are not fully rational, or have goals other than the maximization of expected score, it is very hard to give adequate formula scoring instructions, and the recommen-dation to answer under partial knowledge is problematic (though generally beneficial). Our analysis derives from a critical look at standard assumptions about the epistemic states, response strategies, and strategic motivations of test takers. In conclusion, we endorse the "number right" scoring rule, which discourages omissions, and is robust against variability in respondent motivations, limitations in judgments of uncertainty, and item vagaries.

Website