Why are multiple-choice questions making a comeback?

Often times, resources and pedagogical need are at odds. We are at a moment where a relatively low-resource form of assessment directly taps onto the need for a new skill: slop detection. 

Multiple-choice questions (MCQs) have long been unpopular among some. Often referring to such questions as “multiple-guess,” these critics highlight how incredibly difficult it can be to differentiate a correct response from a plausible distractor. In our current context, this challenge of “multiple-guess” goes beyond MCQ, and is more real-world than perhaps ever before. We are called on to use this very skill — differentiating accurate information from plausible inaccurate information (typically generated by scammers and artificial intelligence) — on a daily basis.  A colloquial term for this is “detecting slop.”  

In academic circles, there is frequent conversation about empowering students to be the “human in the loop” with respect to AI, leveraging skills to oversee AI processes and output. These conversations often jump to the application of skills rather than the building of knowledge that many assume can be outsourced now to AI. This, however, assumes that fact-checking is no longer needed. AI, and humans, get information wrong. Students must still know things to effectively detect inaccurate information.  

Well-crafted MCQs have always been an effective evidence-based way to determine the degree to which students can separate accurate information from plausible-but-incorrect information.  MCQ assessments also allow for the direct development and assessment of “slop detection.” They are rapid to administer, easy to analyze for assessment efficacy (i.e., statistical tools are available to help determine whether items are appropriate), resistant to cheating during a properly constructed testing environment, can be created to assess higher-order thinking skills, and, if used to provide feedback, can help students develop the meta-cognitive skills and knowledge to effectively detect inaccurate responses.  

Administrators and instructors must be mindful that, contrary to popular myth, generating and validating high quality MCQs is not easy and can be very time consuming. The following is a sample of considerations for instructors new to creating MCQ, based on the article “A Review of Multiple-Choice Item-Writing Guidelines in Classroom Assessment,”by T.M Haladyna, S.M. Downing and M.C. Rodriguez, published in the journal Applied Measurement in Education, 2002:  

• avoid trick items 
• use simple vocabulary 
• format response items vertically 
• put the central idea in the question stem 
• use positively worded stems rather than negatively worded stems (unless required) 
• three plausible distractors and one correct response are generally sufficient 
• response options should be placed in logical or numerical order as relevant 
• response options should be similar in content and grammatical structure 
• “none of the above” options increase difficulty and should be used with care 
• “all of the above” options should generally be avoided 
• and use humour judiciously.  

It takes time, care and knowledge of test validation to use MCQ assessment well. Centres for teaching and learning are wonderful resources for those looking for support. Because generating, and then validating, MCQ is so time consuming, instructors should be thoughtful and deliberate in choosing whether test-banks should be kept confidential. Feedback on performance can help learning, but if summative tests are being rapidly generated and not validated, this puts students at risk of sitting a poorly constructed assessment.  

A strategy that may be more helpful is ensuring many practice/no-stakes opportunities for students with feedback within a course (where items can be reused), but protecting summative test-banks used for formal assessment. This allows for the reuse of items carefully created and validated through analysis, and also allows for the editing or deletion of items that do not perform well. Over time, when items are reused/redeveloped, summative assessments can become stronger as a result of ongoing item analysis.  

More than ever, well-crafted MCQs are an effective tool for students to develop skills for slop detection. A well-crafted multiple-choice assessment with plausible distractors that requires students to apply their knowledge and use judgement is very similar to evaluating plausible, but potentially incorrect, content from scammers and AI. Detecting slop is a critical need, and MCQs are an effective assessment tool that can help us to both develop and assess this skill.