Failing our exams

August 24th, 2007

It’s that time of year again, when examination results for the nations 16-18 year old come out, only photogenic girls seem to turn up for their results, and everyone bemoans the fact that either children are getting more stupid (as evidenced by falling grades) or children are getting more stupid (as evidenced by rising grades, which really mask falling standards).

So which is it? When the grades rise, are the exams getting easier or children getting smarter? It’s time for some hardcore statistical fun!

At the face of it, the answer is no-one really knows. Both the rising intelligence and falling standards are reasonable hypotheses that can be deduced from the results, and aren’t even mutually exclusive. Perhaps one is happening, maybe both. Maybe standards are falling in London whilst intelligence is rising in Glasgow.

The key here is this: are these exams supposed to test children against themselves, against eachother, or against a national standard? We tend to think it’s all of these, but there’s a problem in that.

The only way we’d know for sure whether this year’s children were as smart as last year’s is if the children sat exactly the same exam as last year’s kids. But those sneaky boys and girls, they’d find out last year’s answers from their older mates. The rouges.

So what we have is an uncontrolled test. Sort of. Because even though the exam has changed, we trust in those writing it to make it more or less as difficult as the previous one. Don’t we? Well, not really. We wait until the results come back to discern whether the last test was as hard as the previous one. Can anyone see a problem here? We’re controlling the test with the same group of people we’re conducting it on. Wherein lies the problem. The answer is really: this year’s results can’t be accurately compared to last year’s.

So is the point to test the children against eachother? Employers certainly seem to think so. After all, they’ve all received the same education, so each has an equal chance of getting a top grade right? Well, no, because each student has a different background, lifestyle, level of support, and a million other factors that confound equal comparison. Another reason children aren’t truly tested against eachother is because every boy and girl who answers a question right is awarded a point. So the grade boundaries are artificially imposed as “those with >90% right answers: A, those with >85% right answers: B”, etc. This implies that everyone who answers more than 90% of answers correctly must be in the top bracket, but is confounded when half the students end up in this bracket. How can half the kids be in the top bracket? The solution is one of three options: make the exam harder, raise the bracket for the top grade (say, to >95%), or “mark on a curve”. This is a phrase unfamiliar to most Brits, but widely used in the US, where the top 20% of the class is awarded an A, the next 20% a B, and so forth. The problem with this is that the difference between an A and a B student is decided not by his or her ability, but the competition’s. This might seem unfair, but remember: we’re testing these kids against eachother, right?
Maybe not.

Third and finally, are we testing these kids against themselves? Isn’t that the true mark of success to improve upon yourself? I certainly think so. But this is also fiendishly difficult to measure. Every single student would need an individualised series of tests, marking their improvement over the course of their education. Not going to happen.

So what are exams for? Until we can decide, we’ll be confused and concerned about the results every year.

Entry Filed under: General

Permanent Link  |    |    |    |    |