NAPLAN tests are not tough enough for the level of maths students are studying

The latest preliminary NAPLAN results came out recently, but new research has found the test might have little to do with what the kids are actually learning in class.

Our research, presented at the Mathematics Education Research Group of Australasia (MERGA) conference in July this year, found the NAPLAN questions in numeracy for Years 5 and 9 do not cover many of the topics students are studying that year, based on the Australian mathematics curriculum.

Yet the organisation running the tests – the Australian Curriculum, Assessment and Reporting Authority (ACARA) – says NAPLAN should be aligned to the curriculum.

Read more: NAPLAN results show Year 3 students perform better than Year 9 in writing, and it's a worrying trend

New tests, old studies

We found NAPLAN mathematics is nowhere near the year level of study the students are doing at the time of taking the tests. The majority of the Year 5 numeracy test is actually content from the curriculum for Year 2 and Year 3.

For example, these are questions from the most recently released NAPLAN Year 5 Numeracy test.

In the Year 9 NAPLAN test, only 25% is content above Year 7 level. Absent from the tests are some of the more difficult content areas, including use of large numbers to do addition in Year 5 and the index laws (such as x^ax^b=x^a+b) in Year 9.

The tables (below) show that the NAPLAN tests have a year level focus which is much lower than the year level of the students sitting the tests.

Where is the mathematical reasoning?

The NAPLAN Numeracy tests are mainly multiple choice, so it’s not surprising topics that require depth of thinking – such as reasoning and problem solving – are neglected.

This is despite what the lead writer of the mathematics curriculum, Professor Peter Sullivan, says:

[…] reasoning is essential to mathematics, it’s what doing mathematics is about.

Our analysis of 312 questions from the most recent publicly-available NAPLAN tests in numeracy found a significant number of the questions tested recall rather than skills.

For example, in measurement and geometry there is an emphasis on knowing the names of shapes, rather than applications such as symmetry or proof.

We found 19% of what the mathematics curriculum Year 5 students should be able to do was not tested in the Year 5 NAPLAN numeracy test. That figure was 35% for Year 9 students.

The Australian Mathematics Curriculum requires students to be able to sketch graphs, and to explain a plan for selecting a random sample from a population in order to describe the items using statistics. The words “explore” and “investigate” occur in many content areas.

Such aspects of the curriculum are not recognised in the format used in NAPLAN testing. Instead of mathematics being thoughtful and creative, NAPLAN promotes an impression that quick responses to a lot of brief questions reveals a complete picture of mathematical ability.

The move to test online

NAPLAN is moving to an online format, which means tests since 2016 will not be made public because the questions will be reused within the new online system.

In future, students will not all receive the same questions. Instead, questions will be drawn from a secret database depending on how well the students are doing. It will no longer be possible to assess the nature of the questions nor to compare them against the curriculum.

The reliability of NAPLAN is not in question. The questions consistently give predictable results. It is validity that is the issue: does the NAPLAN actually reveal ability at what the students should have been learning?

Read more: Kids learn valuable life skills through rough-and-tumble play with their dads

We say no, as the questions cover material from the curriculum for much younger students and avoid the difficult aspects of mathematics.

NAPLAN is a multimillion dollar exercise which is supposed to assess the attainment of Australian school students. In practice, the results come to teachers too late to be of value for diagnostic purposes, and students are never permitted to know which questions they got wrong.

But NAPLAN is influential in defining the public impression of what mathematics is, and that is not the same as the mathematics curriculum.

If the real purpose of the NAPLAN is to check up on schools and school systems, there has to be a cheaper and less harmful way of doing so.