Confidence Scores for Exam Questions

Confidence Scores for Exam Questions(nomagicpill.substack.com)

9 points by surprisetalk 3 days ago | 7 comments

vmilner 2 hours ago |

I seem to remember some medical related multiple choice tests in the UK use a mechanism of +1 for correct , 0 for unanswered , -1 for incorrect.

bArray 28 minutes ago | |

Typically you have -1/N for incorrect selection, where N is the number of choices. For N=4, you would grade incorrect answers as -0.25.

If you have a person taking an exam that is not confident in themselves or generally knows the subject area, you don't want to negatively impact educated guessing.

CGMthrowaway 1 hour ago | |

A system like that seems especially appropriate for a practice where the foundational principle is "do no harm."

bee_rider 54 minutes ago | | |

Would probably be applicable to engineers as well, or any other field where the practitioner has an obligation to be aware of the limits of their competency.

DANmode 27 minutes ago | | |

and yet.

clickety_clack 1 hour ago |

It would make more sense to just use IRT for grading the responses than trying to add more complexity to the answers themselves.

esafak 2 hours ago |

https://en.wikipedia.org/wiki/Calibration_(statistics)

https://en.wikipedia.org/wiki/Scoring_rule