The department suggests that "[i]f one wants to compare how well the system is doing, one should turn to testing systems like the international TIMSS and SACMEQ programmes, where South Africa has participated for some years."
High dropout rate skews results
A further flaw in using the matric pass rate as a barometer of national performance is that thousands of school pupils drop out long before they reach their final year. The dropout rate is not taken into account in the final pass rate.
For example, when the 2012 matric class started grade one in 2002, there were 1,261,827 pupils. But by the time they came to sit for their final exams, their numbers had fallen to 562,112.
Nicholas Spaull, a researcher at Stellenbosch University who focuses on primary education, says that "students are pushed through the system until grade 10, and then schools realise that if they put these kids through, they are not going to pass grade 12".
"Getting low pass rates in matric is problematic for schools, so they weed out these students."
The 'culling process'
The matric rate is thus bumped up and gives no indication of how the 50% that fall by the wayside are doing. Jansen, in his opinion piece, called it a "culling process" that has left behind half a million people with little or no proper education.
Mary Metcalfe, former head of the Wits University School of Education and a former provincial government minister for education in South Africa's Gauteng province, echoes these concerns. "[The pass rate] doesn't tell us about the large number of children who didn't make matric, who didn't pass grade ten, who didn't pass grade 11 and who failed at grade 12," she said.
The dropout rate has had a significant impact. A 2011 report revealed that "60% of youths are left with no qualification at all beyond the Grade 9 level".
Pupils are choosing easier subjects
Whether as a result of school pressure or individual choice, pupils are increasingly taking easier subjects.
In 2010, 263,034 full-time pupils wrote mathematics. This decreased to 241,509 pupils in 2013. Conversely, numbers of full-time pupils writing mathematical literacy, the easier subject, increased from 280,836 in 2010 to 324,097 in 2012.
The department of basic education acknowledges the impact this has on the final pass rate. "A key factor is the spread of learners across subjects. When this changes, the pass rate can change, even if performance in individual subjects remains the same. In particular, if learners move to easier subjects, more learners pass."
Good performances skew the average
The matric results also conceal the underperformance of the majority of pupils who write the examination. Strong performances in a minority of schools will mask the poor performance of the majority of schools that are judged as dysfunctional.
This skews the average, and does not present a true reflection of the mean for most pupils. This point was also highlighted in Jansen's criticism of the matric results. "[I]f you removed the top 20% of schools - mainly former white, privileged schools - from the national averages, then a very dark picture emerges of a mainly black and poor school system performing far below what the combined results show," he wrote.
Conclusion - Matric pass rate doesn't mean education is on the right track
The improvement in the matric pass rate is good news for those concerned, but it is not a sign that the "system is on the right track", nor that the quality of the education system is improving. An Africa Check report looking at claims made about the 2012 matric results came to the same conclusions.
The matric results are not a good measure of academic achievement in the education system. As the department has acknowledged, they are not designed for yearly comparison or to be a reflection of academic achievement in the education system. The good performance of a minority of schools can also skew the results, as can pupils electing to take easier subjects.
The results only account for about half of those who entered school together. South Africa's high dropout rate means that many young people will never get the chance to write their matric examinations, let alone pass them.