There have been a few comments on the threads on Myschool asking about the other part of the equation – what’s in the NAPLAN tests from which the achievement scores are calculated. Well, if you’re curious, here’s NAPLAN’s website, from which, amongst other things, you can download the actual tests given in the past two years.
I had a quick poke around in the Year 9 tests. As broad-brush assessments of certain aspects of student achievement, in my completely inexpert opinion, they didn’t seem too terrible. The numeracy test was particularly interesting to me – I’d echo some of the comments about the test design made in this analysis of the results in one school – most of the questions the calculator-based test were not made any easier by the use of a calculator.
But what I noticed most strongly – and something that was pointed out in the linked commentary – was the relative lack of emphasis given to data analysis, or in other words the rudiments of statistics. Perhaps it’s just because I’m interested in the area, but it seems to me that lack of statistical literacy is a particular blight on public discourse in Australia. Is it that this test has less emphasis on “data” than actually occurs in Year 9 mathematics classes? Is it just that at year nine level, the other basic mathematical tools haven’t developed to a point where it’s worth starting to teach much statistics? Or is this reflective of the relative emphases in school curricula – and would I be right in perceiving this as a problem?
Links to other analyses of the NAPLAN tests most appreciated!
“Perhaps it’s just because I’m interested in the area, but it seems to me that lack of statistical literacy is a particular blight on public discourse in Australia.”
You should try teaching first or second year social-science students. You’d be horrified — just simple graphs are hard for many (no jokes). I reckon part of this is actually due to what they teach at primary school, and not just the trend away from doing mathematics at high school. In this respect, lots of the stuff you would have done is now cut out (which is why almost everything is done with calculators now), which means that many kids don’t have a good grasp of stuff which you would take for granted and would take you no effort to solve (like, for example, simple inequalities, which a lot of that early learning and memorization stuff helps you to learn).
Given the implied level of abstract thought from some of the problems that I looked at, statistical analysis of data might be a bit beyond the expected students capabilities. That’s not to say that these expectations are ideal.
I’d hope though, that other classes such as science and social sciences would begin work on data analysis from a qualitative point of view.
“lack of emphasis given to data analysis, or in other words the rudiments of statistics.”
I reckon Year 9 students should be able to master the material in Billingsley’s Probability and Measure.
I was going to say something like this in the ‘things everyone should know’ thread.
Two maths techniques that are absolutely essential for getting by in the world are estimation and statistics and so far as I am aware, they are barely taught at all.
You can do all sorts of stats with no ‘advanced maths’ and get some neat experimental results with a spreadsheet. Black and white balls in a bag, regression to the mean, the law of large numbers.
d
Reaching way back into the mists of time, I recall we weren’t taught any ststistics until Matriculation. That may be an artifact of the way it was taught, as it was in the context of elementary bombinatorial mathematics. The next time I encountered it (as a formal discipline) was in 2nd year uni.
Interestingly, my middle son has a pretty good intuitive grasp of probabilities, largely as a result of playing Dungeons and Dragons as a teenager.
David — D&D and other RPGs are excellent beds for developing mathematical intuition across a range of areas. One of my friends has probabilities memorised for anything up to 2 rolls of a 20-sided die (2D20), or four rolls of a six-side die (4D6).
He’s also masterful at “minimaxing”. Give him a number to optimise and a set of rules and he will work out the best combination. He’d have made a brilliant tax lawyer or accountant.
Don’t forget compound interest. It’s probably the simplest concept which is totally misunderstood.
I think it’s time statisticians worked out a good way of making this information understandable to the general mathematically-illiterate population. Stats is becoming a big part of the modern world compared to even 10 years ago, and understanding the basics would really help people to navigate the complexities of a lot of political and lifestyle issues.
It’s interesting to compare Australian/English approaches to maths with Japanese. Japanese consumers routinely use spider-web charts and other complex graphical methods to describe things as diverse as their favourite hairstyle or a hot spring. I have at home a two page fold-out chart of hot springs in Beppu which plots their hardness vs. their sulphur content, colour-coded by region and possibly also sized by temperature, so discrerning onsen-goers can choose the hot spring that suits them – on the basis of 3 or 4 properties of the hot spring – just by looking at the picture. Ordinary people contributing to the magazine Tokyo Graffiti routinely use spiderweb charts to summarise the balance of 5 separate choices at once. In the English-speaking world this is considered to be sophisticated visual represenation of data, taught at university level!
“Don’t forget compound interest. It’s probably the simplest concept which is totally misunderstood.”
The Time Value of Money requires some mathematical sophistication to work (fractional exponents, logs etc), but that’s another great example of useful, practical maths that seems to be ignored.
d
“I think it’s time statisticians worked out a good way of making this information understandable to the general mathematically-illiterate population.”
The thing here is that statisticians (like just about everyone else) have no way of communicating with any reasonably sized sample, let alone an entire population. (sorry, I really couldn’t resist).
Their only opportunity would be if a journalist approached them for, and then they have to hope their clear and lucid explanation makes it through the journalist and the editorial chain into print.
A fun and short book, if you’re interested in this kind of thing, is John Allen Paulos’ “Innumeracy: mathematical illiteracy and its consequences”. NYT bestseller in 1988.
d
The problem there is that the same deficiencies that cause someone difficulties in understanding statistics will give them difficulties in understanding the information regardless of the communication mechanism used. I think that statistics should be like medicine, find an expert you trust don’t try to do-it-yourself.
10 and 11, I was thinking more that statisticians need to start finding ways of explaining the outline of our work to students at high school, not to the public at large and not to enable the public at large to do it themselves. I don’t expect, for example, people with a high school exposure to stats should be able to critique Wakefield’s papers; but they should understand when a professional statistical body claims that the research had a small sample size and a biassed selection process. I don’t claim to know how this stuff could be taught to high school students, mind you.
Stats used to be an esoteric field, but with the growth of computing and data collection it’s becomeing very commonly a central part of debate – witness the MMR, Iraq deaths and AGW debates as examples. People don’t need to be armed to do this stuff themselves, but it would help if some of the outlying concepts were clear to them.
It’s interesting, because we have produced some of the some of the best statisticians going around — Peter Hall (possibly the worlds most cited statistician), Geoffrey Watson (don’t tell me you’ve never heard of the Durbin-Watson statistic!), Pat Moran, Christopher Wallace, William Sheppard, E. Pitman…
And bloody hell, with the bombardment of cricket and Aussie rules stats we get hit with, why aren’t we good at them?
Because they’re rubbish stats, most of the time. Football clubs use a completely different set of stats to the ones the public gets fed, but either they’re kept too secret or journos are too lazy to find out about how they work.
Ditto cricket.
Maybe NZ is an outlier again, but for my final year of high school we had a “maths with stats” course as one of the options. Being geeky, I took that as well as the other advanced maths course. But even dummy maths covered some of the basics (IIRC Lotto was new-to-NZ then and was covered, so they got expected returns and other useful ideas).
It’s hard to know but my impression is that even a lot of geeky types are not familiar with estimation, unit checks and other cheats until they’re taught at university. I recall some horror in my electrical engineering classes when the idea was introduced that estimation errors tend to cancel each other out. Having read Feynman I was already a fan of the idea
I think one of the problems the education system is facing is the growing number of subjects that people are seeing as important for the public at large to be familiar with, while there’s pressure to teach to the lowest common denominator which is a level that can’t possibly include all or even most of them.
First year psych 101 at Uni Melb several years ago was a joint arts science course. I was positively shocked back then at how innumerate the arts students were. It made the very basic statistics bit, which only lasted a couple of weeks, an absolute doddle for the science students, who’d done year 12 maths, and this was just a very basic rehash. I remember one question “what’s the square of a number?”
By the way, the comment on roleplaying or board gaming making for good simple calculators is spot on. When you’ve added up the factors attacking stalingrad versus the unsupplied guard units defending it, plus weather modifiers, plus paras, minus this, plus that, and quickly worked out the ratios a thousand times, you get relatively fast at simple calculations.
I was at the register the other day, bought $8.60 of goods, handed a $10 note over, something happened with the till, and the poor young checkout chick couldn’t work out how much change to give me, had to call someone over to work it out. oh good grief.
desipis: that’s certainly true. You might recall we had a rather lively thread discussing the enormous collection of “essential” information LP commenters would like everyone to know! More seriously, we had a guest post on the national curriculum a little while ago which pointed out that cramming the curriculum with information was kind of redundant these days, where the really important trick is to be able to evaluate the quality of the information you find.
That’s why I reckon statistics is so important – understanding the fundamentals of it a prerequisite to evaluating a large fraction of the information that comes the average citizen’s way these days.
Remember that the NAPLAN tests are benchmarking tests: they are set to discriminate at the bottom of the scale, not the top (rather as IQ tests don’t disciminate very well amongst the profoundly gifted because their focus is on the average person). My elder son’s Year 3 report shows that the middle 60% of students get results starting in Band 3 and going up to Band 5 or 6, depending on subject. If results are above Band 6 they just list it as “above Band 6″. There is no precise calculation of excellence here! I believe there is a set of minimum standards around for each grade.
WRT stats, I don’t remember doing anything beyond standard deviations before Year 11, and the NAPLAN victims are only starting Year 9!
Does that mean the results are irrelevant to anyone with kids not towards the bottom of the scale?
Desipis @20, I believe it’s irrelevant. My son’s NAPLAN results are only useful as a brag point for his relatives; I know he’s performed in the top 20% for everything, but frankly that’s no surprise. Of more surprise to us was his performance in the ICAS Maths and English Usage Exams: top 1% of the state in both. These are tests that do discriminate at the top end of the scale (they’re what the old UNSW Maths Competition has morphed into).
The NAPLAN tests are simply to see if there are people who don’t know the absolute minimum they’re supposed to know for their age. As the teachers’ unions have commented, teachers already know who these kids are. If poor NAPLAN results DO send more interventions to the child, that’s great — but I’d like to see whether that’s happening at all. Anyone seen any quantitative research on that?
In the meantime, we have NAPLAN results being used for off-label purposes, such as MySchool.
And now we can all see which schools have a preponderance of those children, and see if there is anything lacking in those schools. If a school has all the teaching resources available to other better performed schools I’d be very surprised.
If there is one thing parents take seriously it is the education of their own child. If a parent is armed with factual evidence that their school appears to be resource starved they will make a noise about it.
Without evidence they are easily ignored.
I think that we will find that they are easily ignored anyway. They wouldn’t have developed something like MySchools if it meant they had to do something. It is just about creating the appearance of something being done.
As evidence lets see if remote schools in the NT, Kimberley, Pilbara, Western NSW are suddenly plied with money, because it shouldn’t be about the parents demanding action, the NAPLAN results and Myschools tell us where the problems are so the government should just act…
Daryl,
Just to echo what I have found in Uni recently – standing in front of about 600 first year accounting students and trying to introduce the basics of finance was very enlightening. When I asked the (I thought) simple question of “What is more valuable – $100 now or $100 in a year’s time?” the majority put their hands up to say they were the same value. It was only when I suggested that they all give me $100 and I would hand $100 back to them a year from now that they started to get an inkling of the difference.
Tragic.
Robert Merkel @14, I’m intrigued. What are these mystery stats I’ve never heard of?