I don't think it reflects the fragility of the student's mathematical reasoning behaviour so much as their ability to play the meta-game of the test itself. They know they are expected to provide answer, and that they don't know how to reach for it - so they play the odds. I for one have used this technique, exploiting for example the fact that in school, correct answers tended to be round numbers, as an heuristic, for example.

I believe if "insufficient data" was stated as a possible answer (which the student's are not accustomed to), they would state that.

The obvious way to improve this is to make "insufficient data" a somewhat common answer to problems (others have mentioned this). Despite the apparent flippancy, I think this is a deep change, since despite its simplicity, it forces students to think about the problem solving process itself.

I think that, if they were trying to determine this outside the context of "here's a math problem from a teacher-type person", they would realize there's no way to know more easily. But, this still makes an important point, that the way we teach math doesn't look much like how we use it, which raises the question of whether we are really preparing students to use the math they are allegedly learning.

> Yesterday 33 boats sailed into the port and 54 boats left it. Yesterday at noon there were 40 boats in the port. How many boats were yesterday evening still in the port?

Solvable: solution is a range.

All problems become unsolvable if you allow unknowns to creep in: "I have one apple in two hands. How many apples do I have?" is completely unsolvable unless you make assumptions about absolutely everything.

Obligatory link: A Mathematician’s Lament

http://www.maa.org/external_archive/devlin/LockhartsLament.p...

If you want to know how a student will confront a real problem, you need to create a context where they won't try to outsmart whoever wrote the question.

It's entirely possible to solve a math problem using psychology instead of math, because math problems are made by people. We've all done a horrifying amount of schoolwork, and thus have a pretty good data set in our head from which to extrapolate.

Suppose I tell you that an algebra textbook question includes the numbers 32, 4 and 64 in the problem text. What's the answer? I'd bet most of us would guess 2 or 4.