“Whoa!” you say. “Everyone knows that you have a 50-50
chance with two choices. What planet is
this crazy lady from? ”
You are not alone in this thinking. I hear over and over that we have to give
MANY choices per question…and have gotten caught up thinking this way myself in
the past. We think many choices will prove our kids aren’t simply guessing. When
our kids deal with motor impairments, though, too many choices can be an access nightmare. What to do, what to do?
The fact is, with testing, selecting from two choices can
give statistically irrefutable results showing mastery of information. Let’s
look at how this can be…
First let’s look at
what makes a 50-50 chance
We’ll start by backing up to a situation where two choices
DO have a 50-50 chance of being correct. Take a coin and predict whether it
will land with heads or tails up. Now toss it one time and let it land. At this
moment, you DO have a 50-50 chance that it will show heads. Or tails. Agreed?
You could repeat this same random coin toss ten times and about five of those tosses would be
heads.
Try it a hundred times and you should get about 50 tosses coming up tails.
Why is this so? Why do half the tosses give heads and half
give tails?
Because the only factor at play is luck. All the other
factors are equal for each side of the coin on every toss…the balance of the coin, the number of rotations in a flip,
the speed of travel... These weigh equally on the outcome. They won’t change
the probability of a guess based on luck.
If a student is genuinely
guessing at answers, they are strictly shooting based on luck. You could expect
about the same results as the coin toss. They would score about 50% correct on
the answers of a true/false test, a sorts assessment with two columns, or another binary choice test.
When odds are no longer
random
When all the factors to support student success are in
place, luck doesn’t particularly influence the outcome. If the child knows the
answer, has the appropriate access methods, and is caught on a day when health
and sensory issues are met, then a correct answer likely has nothing to do with
luck.
The factors at play—knowledge, access, and health issues—change
the game completely. Testing is unrelated to the coin toss, even when there are
only two possible responses.
When a child is able to score 5 correct responses on a 5-point
true/false test, the statistical likelihood that this was achieved by pure
guessing is about 3%.
But 3% still leaves a tiny possibility the child was
guessing. Three chances in 100. How do we reduce that so we can know beyond a
doubt that the child was demonstrating knowledge rather than luck?
It doesn’t take very
many corrects to get pretty strong data
Five isn’t many questions for a test, but it IS a good
number for a child whose ability to attend to a task is impaired by issues of
access, attention, health, or sensory processing. If you can cycle through a
longer test in short 5-question chunks, you can get some highly valid
information in relatively short time.
5/5 correct can only be achieved by
guessing 3% of the time, as we said earlier.
5/5 done twice (totaling 10/10) gives
<.1% chance--less than 1 in 1000--of guessing as the means for achieving the score. That’s good
enough for me (side note: this would not be adequate if we are talking about
skills needed to fly a commercial airliner or perform neurosurgery, but for
academic skills for most kids, it works).
5/5 done three times (or 15/15
total) only gives a .00003% chance of guessing…and that is statistically
improbable. Most definitely.
The great news is that a child can still demonstrate high
levels of understanding even if they can’t get a perfect score every time.
Let‘s say they can manage to get
4/5 correct on a group of True/False test questions. There is a 16% margin for
guessing, which is too high to put our money on.
If they can get 4/5 again on a
second try (a total of 8/10), the likelihood of arriving at this score by
guessing drops to 4%. Still a little high.
The third time they score 4/5
(raising the total to 12/15), the probability of guessing drops to 1%. That is
1 in 100, so getting smaller!
The fourth time they manage 4/5 (or
16/20 now), the likelihood they are guessing drops to only .4%, or 4 in 1000.
For basic academics, this satisfies my confidence they are exercising skill rather
than luck.
Great news for our kids
with motor impairments
This shows that kids with severe motor impairments who can
only select between two responses really can
demonstrate to us that their answers are intentional.
Obviously, having the motor control to indicate an answer
from THREE choices is terrific for giving strong proof that your answers are
not guesses. It lets you get to that point faster.
For example, in a 5-question quiz where each question has
THREE responses, a perfect score of 5/5 only has a .004% chance of being achieved
by guessing. Everyone can agree that this score is pretty unlikely to result
from luck!
But when motor skills dictate that TWO choices are best, our
kids can still demonstrate their proficiency without the fear of that guessing
has influenced their scores. It just takes longer.
Also, it takes longer if they can’t achieve a perfect score
every time. But it can be done. That, friends, takes a huge burden of pressure
off both the student and his teacher!
A tool to simplify this
statistical computation
Back in my college statistics class, we had to crunch the
numbers to rule out the probability of guessing. On scratch paper. By hand. It
was grueling!
Now, we have a fantastic FREE tool online
that can do this for us. In a
split-second!
It’s so easy to use!
1.
On the first line, type in the "chance" of getting
one problem correct, expressed as a decimal. In a 2-choice problem, the chance
is .5. In a 3-choice problem, the chance is .33. For a problem with 4 choices,
the chance is .25. Kids with motor issues should not be given more than 4
selections per question, so I won’t go farther.
2.
On the second line, tell how many questions on
the test.
3.
On the third line, write how many corrects the
student scored.
4.
The fourth line will tell you the probability that
the student could have arrived at their score by guessing. The tinier the
number, the less chance there is that the student guessed.
Hidden messages: guessing and intentional misses
Scores that are not perfect (or close to perfect) hide
important information for us to uncover.
Those that fall around the 50% chance of being achieved
simply by luck were probably the result of guessing. So why is the student
guessing? Do they not care? That’s OFTEN the case! Do they not
understand? Very possible, but the fact that they haven’t established any kind of
generalization, even an incorrect one, means they have not caught ANY of your
instruction.
Scores that show a statistical probability that answers are actual
errors (say, 0-1/10) rather than guessing are also strongly telling. When you
type the figures into Stattrek.com, you will see that the likelihood of
guessing only errors is a slim as guessing only corrects. There is a message in a page full
of errors. Does the student understand the teaching wrongly, applying
the concepts backward? It’s very possible. Is the student trying to tell you
she is bored? Hmmm…we need to pay attention to this.
Are you feeling more confident now in providing tests that
ask students to select from just two choices? They can be devised to give you
very accurate feedback about the child’s understanding.
The key is in the number of repetitions, but even that can
be surprisingly small. Once you start crunching numbers, you can see
that tests don’t need to be pages and pages of questions. Go small, add more if
needed, and check back with Stattrek.com often to see if you’ve eliminated the
probability of guessing. You saw from our examples how a perfect score on only
15 true/false questions is statistically irrefutable as being skill rather than
luck!
Let me know if you still think I’m a crazy lady from another
planet or if this makes sense. Just leave a comment in the box below…I read
them all!
* * * * * * * * * *
You might also enjoy reading:
You are invited to join the conversation between Peggy Ployhar and myself on October 1, 2019 at 8pm (Central Time Zone). We'll be talking about ways you can teach your nonverbal children and teenagers. This conversation is directed at families who homeschool but is relevant to nonverbal learners in the school system as well. Bring your questions about adapting instruction and evaluation and we'll give them our best go!
