According to Julia’s press release, the MySchool website is supposed to tell whether your local school is working well:
My School contains important information about each of Australia’s 10 000 schools including the number of students at the school, the number of teachers at the school and how the school is performing in national literacy and numeracy testing.
Parents and school communities are also able to compare their school’s results with neighbouring schools and up to 60 statistically similar schools.
By comparing statistically similar schools, it will be clear which schools are doing well and which schools need an extra hand. By comparing statistically similar schools, it will be clear which schools are doing well and which schools need an extra hand.
The very idea of making this kind of information public is of course controversial. If it is to be done, however, you’d hope that it’s done in a rigorous way, so that schools are indeed compared against appropriate benchmarks.
So, like half of Australia, I plugged the name of my old high school into the system to see how it rated. And I was shocked to discover that Tallangatta Secondary College had bands of red all over the place, indicating underperformance relative to “similar schools”.
These “similar schools” were identified according to the scores on a specially developed index:
“The Index of Community Socio-Educational Advantage (ICSEA) is a special measure that enables meaningful and fair comparisons to be made across schools. The variables that make up ICSEA include socio-economic characteristics of the small areas where students live (in this case an ABS census collection district), as well as whether a school is in a regional or remote area, and the proportion of Indigenous students enrolled at the school. It has been developed specifically for the My School website for the purpose of identifying schools serving similar student populations. The average ICSEA value is 1000. Most schools have an ICSEA score between 900 and 1100. ICSEA should be interpreted with the assistance of the About ICSEA Fact Sheet, ICSEA Technical Paper and relevant FAQs.
If you want the gory details of how the ICSEA is calculated, you can have a look at the technical report here. Essentially, it’s a weighted composite of a bunch of demographic indicators about the communities the schools draw their populations from. Nothing about the demographics of the actual parents, just the communities.
The upshot? According to ICSEA, among the schools my little rural high school most closely resembles are MacRobertson Girls High, Sydney Boys High, and Sydney Girls High. Yep, that’s right. Selective government high schools that skim the most talented students from other schools around an entire capital city, or even across the entire state.
While these are of course three extreme cherry-picked examples, it seems to me likely to be representative of a broader problem – comparing schools that for whatever reason select a subset of students over a wider geographic area, to government schools that are the default option for students.
As such, my strong suspicion is that on this basis alone the information is dangerously misleading in its present form.
Elsewhere: Mark points to a report from the Grattan Institute, which suggests that “value-added scores” – that is, measuring the average progress of individual students – would be a much better assessment of the contribution made by a school. Sounds like a better approach, though it’s still a bit orthogonal to the problem I’ve referred to here.
A similar scheme has operated in Tasmania for the last two years, the sky hasn’t fallen in, there hasn’t been a mass exodus of children out of lower ranked schools, and parent-teacher meetings work of a more informed and less anecdotal basis.
I’d be a bit wary of engaging in a knee-jerk defence of the teachers’ unions on this issue.
I’m not. I’m merely observing that the statistical basis for “comparing similar schools” appears to have significant flaws.
I was shocked and appalled to learn that 50% of the schools in Australia are below average!
Rob, did you see the criticism from the Grattan Institute? http://www.grattan.edu.au/blogs/?p=23
Their position, which makes sense to me, is that what should be published is longitudinal data showing improvement (or not) among cohorts of students, as a better measure of what influence the school actually has on outcomes.
Christopher Pyne opposes MySchool, which suggests in my mind that it may have some merit.
No, I hadn’t, Mark. But it does, at first glance, make sense to me.
On a related point, I did wonder what the point of presenting Year 7 scores for high schools in Victoria was, given that it’s much more reflective of the student’s primary school education than their high school education.
The demographics clearly didn’t include “users of the site with red-green colour-blindness”.
Shockingly, breathtakingly shoddy work.
@6 – Rob, yes, I haven’t read the report itself, but the presentation of the argument in yesterday’s media seemed pretty persuasive to me.
I’d also observe that the Grattan Institute, when it was established, was represented by some (including Crikey) as some sort of Kevin Rudd front. Perhaps that criticism was misplaced.
Rudd’s and Swan’s old school is pink-banded. As is Lindsay Tanner’s.
My old public high school, Normanhurst (in Sydney), is now a selective boys’ high school with a high proportion of students from Asian ethnic backgrounds. According to this site it is ‘statistically similar’ to (inter alia) Adass Israel School, Elsternwick VIC 3185, a school for orthodox Jews, and Ascham School Ltd, Edgecliff NSW 2027, an elite girls’ private school that takes in boarders.
They might be statistically similar but I can’t think of any sense in which they are similar for any practical purpose. Why comparisons of academic outcomes are supposed to be meaningful is beyond me. If this is the kind of metric that education ‘reformers’ want to use as the basis for paying either schools or teachers or both, it simply reinforces the stupidity of the whole idea.
Mark @ 4 – There is the transcript of an interview with one of the institute representatives on ABC radio here. Says the website is a step in the right direction, but needs improvement.
I’d agree with the importance of longitudinal data – the more detail the better as far as is possible whilst still keeping anonymity. But people will still need to be careful analysing the results as some schools/teachers may be good at improving struggling students but not as good at improving already well performing ones, and vice-versa.
Its not as simple a matter of “best school practices”, but school practices (and more likely specific teacher styles) which work best for a specific child.
Robert – I think the problem you describe has been one of the criticisms of the way the federal funding for schools works too. Instead of being based on the students who actually attend its based on averages on where the students come from which can be quite different.
Mercurius – 50% below the average or 50% below the median?
/stats pedant
Razor: if it’s normally distributed, they should be close enough to equal. Gotta love the Central Limit Theorem
Well I was suprised to see my alma mater had lots of reddish bars, given that Razor Junior is off there for Kindy in 2011 whether he likes it or not. They have time to improve.
However, I was happy to see that the Razorette’s school is covered in green bars. Money well spent and backs up our anecdotal experience so far.
I agree that it is not a perfect system for comparison, but it is a start.
I think that being able to demonstrate value adding is important.
I would like to see comparisons of parents’ attitudes towards education also as a factor.
I think your point is well made Rob, especially with respect to schools that draw their student population from a wide variety of geographic locations. Indeed, the technical report you linked to mentions that the index values had to be hand tweaked to deal with boarding schools (not so objective one might think…).
However, we do know that this is exactly the sort of analysis that goes into allocating government education funding. Which is possibly more of a concern than the publication of the said analysis on a public website (although as you might note that issue is orthogonal to the league table/teaching to exams criticisms).
Indeed, Fmark.
One benefit of the transparency should be to demonstrate how ridiculous the current school funding model is.
So, the comparison is of the NAPLAN performance of, say, one cohort of year 9 students with the average performance of a random sampling of 60 year 9 cohorts from areas with similar ICSEA scores?
Isn’t it a problem that kids start school at different ages in different states, especially for the primary school results?
Is it actually possible to compare, for instance, this year’s 9s with last year’s 9s in a meaningful way? Does NAPLAN control for “year of smart kids” and “year of dumb kids” to reveal poor teaching?
And isn’t the name a bit ‘off’? I’m not comparing schools, I’m comparing how the local bunch of year 7s measure up against their “peers”…
Then again, trying to sell “Are Our 5th Graders’ Grades As Great As Their 5th Graders’ Grades?” seems a tough ask.
My old school, Geelong Grammar, has a lower ICSEA score than other similar private schools because it draws it’s students from a wide geographic area (primarily a boarding school) despite it being the most expensive school in Victoria, if not the country. I’m sure Kangaroo Ground Primary School is a bit peeved at being statistically similar to a school that has more non-teaching staff than they do students.
I agree completely. The problem with longitudinal data, or with getting enough sample of a schools year 9 class to alleviate the smart year/dumb year factor, is that it takes a long time. Do we really want to accumulate 30 years of data about a struggling school before we do anything with it?
The issue of whether the data does actually recognises a struggling school is another matter, as Rob points out. In the cases of clearly struggling schools, you don’t need the data to tell you there is a problem. Nor does the existence of a problem seem to be associated with an injection of funds…
Good pickup.
“Mercurius – 50% below the average or 50% below the median?
/stats pedant”
If the distribution is symmetric, they are equal. The test scores are probably standardised so that the distribution is symmetric.
/stats uber pedant.
All this bitching about how including selective schools invalidates the whole exercise is ridiculous. Of course those schools aren’t comparable, but they are so few in number they can be safely ignored.
I did a bit of league tabling and what I found interesting was how a lot of private schools did not do so well compared to their public school peers. You reckon spending all that money sending your child to Old School Tie Grammar is going to get them a better education than your Middle Class High School? Not on this evidence.
Of course both Middle Class High and Old School Tie Grammar do better on the tests than Working Class Full of Non English Speaking Refugees High, but we knew that already.
And as for sending your kids to a private primary school, well that is a complete waste of money on the test evidence.
Still struggling with spotty MySchool website performance.
Looked at one local primary school which was above average almost across the board, but fell down because the students were not good story tellers in 2008-09.
Well. Now we know who won’t be producing a Booker Prize winner amongst their alumni.:-)
Was also facinated to see that this NSW North Coast school was supposed to be within a few kilometres of both Syndey’s inner and extreme outer suburbs, according to that same website.
I would have thought that the wide postcode variations would have given a bit of a clue to whoever prepared this particular webpage, but apparently the data was entered by a troop of monkeys practicing for their next attempt at Shakespeare’s sonnets.
Just looked up my nephew’s school, Rouse Hill Anglican, and it is supposedly worse than the State school my sister moved him out of.
According to the news on the way home, Myschool is so competent that it tells Tasmanian parents that Bicheno primary is near other schools on the north-west coast, and Cressy High School is just a couple of km from Zeehan High. You can have fun with google maps to work out just how geographically erroneous this is, or just take my word for it.
And as the head of P&F Tas noted on the radio, the scores are just a snapshot in time – what value is that?
Day 1 and already Myschool is incompetent and inaccurate. Hooray
The head of P&F Tas is wrong. There are two years of data, 2008 and 2009, so it isn’t a snapshot in time. In another 5 years there will be 7 years of data which will be enough to see how a school has been trending.
When you look for local schools in Albury North,postcode 2460 you also see Cessnock schools postcode 2325.
Albury is 550km south-west of Sydney
Cessnock is 150 km north west of Sydney
Aaargh! It makes running a bog-simple query so… bleedin… hard!
So, I just checked the site out as well. My old school – Reynella East High School – appears to have an index value close to the national average (997).
Taking a look at how the index of community socio-econoimc disadvantage is calculated the methodological problems are pretty clear.
Take this statement “The best way to compare the academic performance of schools is to find groups of schools with students of similar abilities on commencing school. Unfortunately, no such measures of starting abilities are currently available nationally, so instead, attention focused on finding proxy measures that are highly correlated with student performance.”
So, the sutitability of the measure is going to be determined by how well the index correlates with student performance.
Unfortunately, the index is almost entirely based on the characteristics of the local area that the student population is drawn from, rather thant the characteristics of the students and parents themselves.
As Robert says, that leads to problems when schools that are not selective are compared to those that are (a particular problem in NSW). That selection of course doesn’t have to be only on the basis of academic achievement. You will also get problems when parents that care more about their children’s education self-select into schools.
In short, there are likely to be all sorts of omitted variable and endogeneity biases in their equations that help them identify statistically similar schools.
It is also quite disappointing that while they show the statistical significance of the individual variables they use, they don’t show the overall explanatory power of the regressions or any tests of the appropriateness of their methodology. The appendix is a waste of space.
For parents the statistical comparison is made even more useless by the fact that they are unlikely to know anything about the schools their own is being compared to. For example, REHS is compared to 50 schools, but only 2 are in South Australia – and both of these are country schools (REHS is outer metro)!!
The Grattan Insitute report is bang on in suggesting that schools should be benchmarked on value add. The problem the government faces is that they don’t have the necessary data and infrastructure to make such comparisons yet. So, they have proceeded with the next best thing.
Unfortunately, it is an example of a little information being both dangerous and highly misleading.
Does this ICSEA thing remind anyone of anything? SES school funding model? That was a complete failure in terms of identifying needy schools and fixing problems, too.
Trevor Cobbold in Dissent magazine says: “As far as education policy is concerned, the Rudd Government has given John Howard and David Kemp another term in office.”
Numerous attempts just now to actually locate my son’s HS via the one and only pathway in have failed. No region locator, no list of alphabetised schools by state. What a joke of a front end website. Pathetic.
“The head of P&F Tas is wrong. There are two years of data, 2008 and 2009, so it isn’t a snapshot in time. In another 5 years there will be 7 years of data which will be enough to see how a school has been trending.”
It will tell you how the school is trending eventually, but this doesn’t really tell you enough about school performance unless you can be certain that the characteristics of the student body are unchanged over the period. Unfortunately, the methodology will only tell you whether the characteristics of the local areas students are drawn from have changed, not whether the characteristics of the students themselves have changed.
Surely you can see how this creates the potential for bias?
Think of it this way. The methodology effectively assumes that the student body at a particular school is a random draw from the local areas that student body lives in. But why should that be the case?
I’ll give another example where the site is misleading (espcially when reported in the news).
The Australian has a piece comparing public and private schools and goes on to say for example that Riverview does worse in reading and writing than statistically similar schools at year 9. What they don’t say is that the website matches Riverview largely with primary schools only, so the sample of secondary schools for comparison is reduced to 17! Of those, for year 9s it actually performs above 10 of them. However, it is rated below average because 1 school (Sydney Grammar – a selective school) drags the average up substantially!
Surely the people that put this website together realise that when the sample of comparison schools is low that the use of the mean rather than the median could be substantially misleading?
Btw Sam, this is actually an example where including selective schools DOES bias the comparisons the website was designed for.
One of the interesting things I noticed is how well many of the really expensive private schools wern’t doing. I was surprised that schools like Carey arn’t doing much better than their elite peers (indeed they’re doing worse — too bad if you’re paying $25k to send your kiddies there). Alternatively, if you look at some of the elite public schools like Melbourne High, they’re doing much better. Even the almost top tier public school near me (McKinnon High) is doing better than them — obviously it pays to buy into areas where there are good but zoned public schools.
“Even the almost top tier public school near me (McKinnon High) is doing better than them — obviously it pays to buy into areas where there are good but zoned public schools.”
Actually, until we know what the starting position of those schools is (perhaps the average starter at Melbourne or Mckinnon is brighter than the average starter at Carey?) we won’t know whether your supposition is correct.
I just checked my daughters priamry (and surrounding schools) and one thing I noted was this: the large variance in *every single result* between 2008 to 2009.
Suggests the indicators may be unreliable; certainly for comparative purposes. Makes me question the value of the site – even though I’m fairly neutral on the idea of the exercise.
My only real hope for the thing is that it will raise awareness of the necessity for a new funding model based on need, actual socio-economic composition (not imaginary, based on postcodes, as present) and performance potholes.
@12 Razor you’re right and no you’re not being pedantic. I remember my Year 9 maths well enough to know that the 50% applies to a median, not necessarily an average. But it was a joke, Joyce, and it’s just a figger of speech, innit.
“My only real hope for the thing is that it will raise awareness of the necessity for a new funding model based on need, actual socio-economic composition (not imaginary, based on postcodes, as present) and performance potholes.”
Precisely the point of the whole exercise, right from the start, no fool our Red Fox, believe it. Teh Government is treading water on the “lift all boats” mantra for the first term, while collecting and publishing the data for “evidence based” policy (and batting off the AEU etc, feeding cheap fodder to Chris Mitchell, and forking the Libs, esp Chrissie Pyne), and when the shouting has died down, to be followed by a full scale assault on Howard’s ugly, divisive, inequitable and unaustralian funding model, with the new model ready for unveiling by 2013…meanwhile, I am totally enjoying the website surfing experience, what a truckload of delightful information…
I also suspect thats the point of the exercise, Grace. Nothing wrong with making public policy, like, public. A lot of the nasties introduced by Howard had to be on the hush-hush, or under conclusive certificate – as they wouldnt have passed open scrutiny.
I send the kid to a local government primary in semi-inner-city Melbourne. I am surprised to see that Geelong and Scotch Grammars score a lower Socio-whatever-index than our school, despite their school fees being so high, and hence saying something about their parental demographic. The bottom line is that the “statistically similar” comparison is not based on actual student composition of the school in question, but on the average demographics of the area where students are drawn from.
I think we can all agree that putting selective schools into a comparison group with non selective schools. I don’t think it makes a big difference statistically but it is such an own goal that whoever made that error should be given six cuts of the cane.
LO, you ask why we why we should assume that students at a school socio-demographically representative of the area where that school is. It is a reasonable question. It some cases the assumption is clearly false. Sydney Grammar, for instance, which is located in the Sydney CBD, draws students from anywhere in principle, but in practice from the wealthy suburbs of Sydney. The same is true of most of the big name private schools. But you can still compare them with each other.
But for zoned government schools, it seems like a reasonable assumption. Why would it not be true?
Sam, the problem is that a large proportion of the comparison schools often turn out not to be zoned public schools, and so biases creep into the exercise. Also, in many parts of the country zoning for public ends up being somewhat elastic through specialisation. So, for example, some public schools specialise in music and are then allowed to draw people from outside the zone into the school. In SA, for years public schools in good areas have been able to take in students from other zones that are academically bright and whose parents want them in a better educational environment. In addidtion, when some parents elect to send their kids to private schools, unless that process is random, which it isn’t, it will drive a wedge between the average characteristis of a local area and the students drawn from it going to the local school
Again, simply put, there are lots of reasons why, even amongst publicly zoned schools, the census data may not be representative of the student body at that school.
While I agree 100% that longitudinal comparisons are superior (the trends matter more than the levels, in other words), my completely stats-ignorant self wonders if there is already a crude approximation – because it lists 2 or 4 years, and assuming most kids attend for the duration, wouldn’t comparing the scores give you an idea of how well a school “value-adds”?
For example, my own school, Telopea Park in Canberra (though I only went for highschool), shows its Year 5s doing worse than its Year 3s, and its Year 9s doing worse than its Year 7s (relative to Sim and All) in both 2008 and 2009.
It’s good for a laugh LO. I’d be cheesed off if I was paying 25K a year for a school and they couldn’t get above their demographic average, no matter what the difference in intakes are between schools (and I can’t see why Carey would have a worse intake than the zoned McKinnon unless variance is really not on their side for those years, although I agree the comparisons are not unproblematic — for any of the schools).
Conrad @34: “Alternatively, if you look at some of the elite public schools like Melbourne High, they’re doing much better.” Yep, I noticed the same thing, for the schools that I knew, public and private.
Perhaps money doesn’t guarantee brains? But it does buy you more dedicated teachers. This effect would show up better in longitudinal studies.
I saw a Scientific American report some years back, that indicated there is only a moderate correlation between academic ability of the parents and their kids. There is a “mean reversion” effect, where bright parents may have average-ability kids, and average parents can produce potentially gifted kids. Genetic variability at work?
Apparently, to some degree, giving a rich learning environment (plus home tutoring, etc) to average-ability kids can lift their performance, relative to what might happen in an empoverished learning environment. However, you can’t turn them into geniuses.
The biggest effect was on the potentially gifted kids in the empoverished environment. This would be where the longitudinal studies will yield fascinating results for creating a rich learning environment in more of our schools in the future.
Perhaps selectively drawing from a larger pool of candidates (elite public schools) can yield a larger cohort of students with outstanding potential? These may then feed off each others ability in class, to generate outstanding results?
This all could be great news for parents who don’t have the money to afford private schools for their kids. Previously, without the comparison tables, the working assumption was that cost = quality.
We all tend to make that mental shortcut (cost = quality) when buying equipment that we don’t understand. Unscrupulous marketing execs know this very well. Those that know how to read the specs can work out whether they are really getting quality for their money.
Now parents have some kind of spec sheet for schools, to make a more reasoned judgement, if they are so inclined. I bet the school principals will find themselves bombarded with demands from concerned parents that their weak areas are improved. Is that such a bad thing?
I had my doubts in the beginning, but having poked around in the site, it looks like the beginning of something which could be very useful, if used wisely.
LO, I don’t think a few out of area music students are going to change the average student demographic in a school. Your point about selectivity in private school enrolment is more serious. So we might have a situation where the poorest half of the population in an area sends their kids to the local government school, and the richest half sends their kids to the local private school. Then the socio demographic of the government school will be not as high as the census average and for the private school it will be higher. Fair enough. But how big is this problem in practice? And what is striking about the test results is how similar a lot of the state schools are to geographically adjacent private schools where the parents pay big bucks. If it’s true that the state schools have a less wealthy parent pool, then that reinforces the lack of value in the private schools.
I looked up my old high school – a rural private school – and found it solidly in the green. This is no surprise, as it’s the only non-Catholic private school in the area and picks up most of the wealthier families (at least until Year 9 when they send their kids to board at more expensive schools in Melbourne) and gives out scholarships, too. Weirdly, though, it has a noticeably *lower* ICSEA score than the public high school in my current small town (which, unsurprisingly, has a lot of red). This is not a wealthy town by any means, and has a relatively low proportion of students continuing to VCE or further education. Maybe they’re counting farm property values, which is extremely skewing!
What Elise said.
To me, the big take out, as also noted by Conrad, is that a lot of the private schools aint that great, and especially not when you consider what they cost.
Those who think this is a betrayal of public education by Julia Gillard (a sentiment given splendid expression in the new thread) have got it backwards. By putting up the data, she has exposed much (though not all) of the private school sector for what it is, an exercise in marketing with little substance.
I saw some research at a conference in England 2 years ago which claimed that the longitudinal data sets don’t give any useful information either. Under a properly constructed multilevel model, the confidence intervals around measures of performance between year 8 and year 12 were so wide that the measure basically only enabled one to distinguish statistically between the bottom 3 or 5 schools and the top 3 or 5. Everyone else was within everyone else’s confidence interval. The conclusion of that research was that the whole process of using longitudinal research to inform parents of the likely outcomes of enrolment in a given school is pointless. And that was in England, with a much larger sample size than Oz. And after adjusting for family-level characteristics.
In fact, in that session the general finding was that family-level characteristics don’t add much to the analysis. I think (a few famous examples aside) this is a common finding about complex multi-level models – they don’t improve on the basic stuff.
This was a presentation at the Research Methods festival of the ESRC, too, by the top schools researchers. I think these websites are an education version of the kind of statistical hocus pocus that led the financial markets astray. I guess it’ll be abandoned in a decade or so…
Sam, I was just using that as an example of how there is a lot of fluidity between apparently zoned schools.
And Elise, having schools bombarded with demands from parents for improvements in particular areas is a good thing, as long as it is based on the right comparisons – what they are doing with what they have been given.
Statistically, this system is not a robust way of providing that information.
One more point – education systems are not just about maximising performance on standardised tests – there is a values element as well. We all have our different views on how values should be addressed within an education system, but at the very least we should acknowledge that many parents send their kids to private schools for reasons other than that they maximise test performance.
At this stage, I’m leaning toward Grace and Lefty E’s point of view.
What the site is publishing is just data at this stage, the critical point is what happens next, and I don’t think the govt has explained that very well. For that matter, they haven’t explained what the data on the site means very well, either.
It’s almost impossible to understand without reading the ICSEA Technical Paper.
The critical thing to understanding what the site is about is that Index of Community Socio-Educational Advantage. The index has been constructed to be the best predictor of school performance, based on whatever data the ABS had available about the Census Collection Districts (groups averaging 220 households) where each and every child lives.
They chose the 14 variables with the highest correlations, and which had statistical significance at the 99% level. The variables aren’t really all that surprising, and are shown in Table 1 in the paper. They include things like percentage of households earning greater than $52,000, percentage of unemployed, percentages of people qualified with less than high school, high school, certificate and diploma, percentage of indigenous people, percentage of single parent households.
They’ve fitted an equation that predicts the NAPLAN results of a school, based on the characteristics of the neighbourhoods where the students live, and the equation has an R-squared value of 86.1% (which is very good).
From the selection of variables that are the best predictors of performance, it’s obvious that disadvantage is good predictor of poor performance, but that advantage above a certain minimum (e.g., incomes much more that $52,000, or degrees above diploma level) wasn’t a good predictor.
Using the equation, it’s possible to identify schools that are performing better or worse than whould be expected, given the background of the students. One example is Brewarrina in NSW, which I came across while idly fooling around with the site this morning. Brewarrina is a K-12 school with 97% indigenous students. The school’s NAPLAN results are way below national average, but at the same time, they’re way, way above what would be expected from the index. They seem to be doing something right, (e.g. perhaps there’s extra funding from the NSW govt, or there’s something special about the teachers or community there, etc), but whatever it is, it’s working and maybe should be replicated elsewhere.
SJ
I’ve read the technical paper and found it far from convincing. For a start, it doesn’t tell you how the school performed controlling for the background of the STUDENTS, it tells you how the schoold performed controlling for the background of the people living in the local areas from a school’s students were drawn from. These are not the same thing. There will be some schools for which such a comparison is meaningful, and others for which it is horribly misleading.
Don’t get me wrong, I like the idea of publishing information about school performance, but I think it is important that the information is not misleading.
I was concerned about this, but if the end result (thanks for the thought, grace @ 38) is that the govt says, “Why are we pissing so much money away on these private sector fuckers?”, it’s all good.
Try again with the link to the ICSEA Technical Paper.
LO Says:
What you say is correct, but the ABS statistics allow for anonymity, and schools aren’t allowed to compel parents to supply the same data.
This will cause some anomalies in some cases, but when “local area” means around 220 households, not so much. The paper actually identifies some of them, where the indigenous population is mixed in with relatively well-off cockies who send their kids to boarding schools.
My children’s school scored slightly above average on the social economic scale,(they did quite well on the coloured in bits!!) with just 1% of the population of our town being in the bottom 1/4, I figured that since that was probably me (a full time student on austudy) that I must be dragging down our socio economic ranking then I might pull my children out and enrol them in the better performing school over the hill which apparently has o% of parents in the bottom 1/4 and 60% in the top 1/4… I could apply for an ‘out of zone’ enrolment surely???? Based on the fact that their school did better than ours??? Based on the fact that Julia said I could transfer my kids if I wasn’t happy with the way the school was performing. I think I’ll go and fill out an enrolment form tomorrow!!!!
What a load of rubbish….
Every child is unique and learns at a different rate, has different abilities, strengths and weaknesses, some parents fall in to the bottom 1% of socio economic groups but have a child whose results are off the radar in the glorious naplan test… where does that leave this data….!!! everyone roughly knows the socio economic status of the schools in their area! All we really care about is how our children are going, if they’re doing well then all is good, if they aren’t, reading a league table isn’t going to help them one iota unless the government puts their money where their mouth is and provides real support for children at risk of falling through the literacy and numeracy gaps. I find it all a bit of fun and can’t help but feel slightly smug about the in laws who’s kids are at Carey Baptist Grammar and .. well … that school just isn’t doing as well as they’d like… a little bit of pink on that graph! … but a very high socio economic ranking!!
The problem is much bigger than that SJ – any school that draws a large proportion of its students non randomly from a given local population will not be measured well in these comparisons. It is a classic case of selection bias.
This is correct, but sort of irrelevant.
The My School stats for the NSW public selective schools that I’ve looked at (and this isn’t supposed to be any kind of definitive pronouncement here) shows that academic selective schools have NAPLAN results above index expectations, and that sporting selective schools have NAPLAN results below index expectations.
This should be no surprise.
Questions about the desirability or otherwise of different kinds of selective schools aren’t addressed and don’t seem to me to be intended to be addressed by the My School data.
The idea that this is all a wily scheme to make people amenable to more equitable education funding warms my heart. It’s wonderful that a few true believers still cling to the belief that Rudd Labor has a commitment to social justice. Naive and a little troubling, but heart-warming nevertheless.
Any changes to education will be designed to propitiate the great gods Efficiency and Productivity. The end.
mercurius @ 37 – having been bashed around the ears and had dusters thrown at me over Normal Distributions and CLT I can see not many can take a stats dig in the manner it was meant.
Peace brother.
sg @49: “Under a properly constructed multilevel model, the confidence intervals around measures of performance between year 8 and year 12 were so wide that the measure basically only enabled one to distinguish statistically between the bottom 3 or 5 schools and the top 3 or 5. Everyone else was within everyone else’s confidence interval.”
I’m not sure if I understand you correctly. Are you saying, more or less (in layman’s terms), that the improvement in average (or median) performance of a Yr 8 class to a Yr 12 class is roughly similar for a wide range of schools? If so, then that would not be a great surprise. You may have a wide range of potential ability, and movement of averages doesn’t show up as clearly as movement at the margins.
I was thinking about longitudinal development of ability for individual children. That is obviously a big ask of a national system, and is probably prohibitively expensive to operate on a continuous basis. Never mind any privacy issues. As such, perhaps it might be better to look at changes in the first quintile and fifth quintile, for example?
To draw a long bow, a slum suburb which has started to be redeveloped/modernised will not show changes in the median house prices for quite a while. However, you should see changes in the minimum and maximum house prices relatively quickly.
There is no doubt that the Myschool system will have troubles with classifying some schools satisfactorily. If I think back to a primary year spent in an outback NT school in a very small community, we must have had about half aboriginal kids from the general district, together with kids from families working at the mine. What the hell demographic is that? What is its benchmark?
We had most levels of primary school in one room – the teacher spent most of his time trying to maintain some sort of control, with limited success. Total mayhem, actually. Sweltering humid heat in monsoon season, dry red dust and flies otherwise, and no facilities except blackboard and chalk. He tried to give different work to kids at different levels, but his focus was understandably diffuse in trying to cover all bases at the same time. In retrospect, I have to admire his efforts in even trying to teach in those conditions.
Some of the kids didn’t even have a proper home to go to in the evenings, never mind motivated parents to drive them to a library for material for assignments. Homework was basically optional. Some kids’ families went walkabout in mid-term, so their learning suffered significantly. A longitudinal study on progression from Yr 8 to Yr 12 in this school would be a joke.
I doubt I learned anything academically that year, but I did get an early appreciation of the impact of unequal learning environments especially for aboriginal kids.
We could dismiss the Myschool system on the basis of these types of argument. However, the sea of red in the metrics for that type of school, with this new system, would stick out like a beacon. Those kids should then get the attention they needed from the system. A comparison doesn’t have to be accurate, to be useful. If it is used sensibly.
I just cannot see how you think it is irrelevent. The myschools website does not just compare zoned public schools with other zoned public schools. It compares them with both selective private and public schools as well. Its colouring scheme simply compares the school’s performance with the simple average across the comparison schools. This in inappropriate in the presence of selection biases. In addition, the clear non-normality of the distribution of performance within the groups of 50 schools makes mean based comparisons problematic. They should at least also present the comparison with the median. In NSW around half of secondary schools apply some sort of selectivity (not always acadmic) to student enrollment. That significantly reduces the usefulness of the comparisons for parents with students in such schools.
Ken Lovell @59: “Any changes to education will be designed to propitiate the great gods Efficiency and Productivity.”
Perhaps you are right. Efficiency and Productivity.
Perhaps the best way of achieving greater overall productivity is to fix the poorer learning environments?
Warm and fuzzy outcome, for all the wrong reasons?
Why are so many commenting here surprised that private schools often don’t compare that well academically with state schools? My experience of a lot of families who opt for private schools when they can afford it, and sometimes even if they really can’t, is that they know their kids are struggling and feel they might get them an edge by paying for them to be in smaller classes. I’ve rarely seen star pupils moved by their parents to a private school.
Selective schools, both state and private, are in a different category of course. Some parents are so anxious to push their kids academically they have them listed at a private school at birth just in case they don’t get into a selective school. These sort of parents don’t understand how just scraping into a selective school is no guarantee of academic success in the long run. It may save them private school fees but even in a selective school at least half the children in that school are going to feel mediocre or worse. It could be that an average performer in a selective school could be among those at the top of the class in their local school and happier with neighbourhood friends.
Staff pupil ratios and school ethos tend to ensure that kids seem better behaved in private schools and that’s what many parents want more than anything. A lot of migrant families whose English is not that great make the mistake of sending their kids to private schools for this reason and to give their kids a better chance than they had. This is initially socially difficult for the kids and later creates problems within the family as teenagers’ English improves and they integrate more with their peers and lose respect for parents.
I wonder how these and other important aspects of education are presented on the My School website. I’ve always had reservations about Julia Gillard having the Education Portfolio. She never seemed to me to understand that education is not just about academic achievement. She’s still a great Deputy PM!
Elise, my memory of the presentation was not that there are no differences between schools – these are huge in England – but that attempts to predict student performance in a school over a four year period, after adjusting for the characteristics of their family, the school and the area they live in flounder on the statistical uncertainty.
This essentially means that the record of a given school in getting year 8s to be successful in year 12 (the 5 year longitudinal achievement statistic) is not useful for parents attempting to use league tables to place their year 8 students in a school, after adjusting for the area and family of the students origin. You might as well just roll dice.
On an aggregate level though the powerful effects of poverty remain noticeable (in England). I think there is a story in here about the relative value of statistical modelling for identifying broad cross-sectional differences, vs. using statistical modelling for prediction over long periods of time or for narrowly defined sub-populations. Which is why I threw in the comparison with statistical modelling in finance.
All of this reminds me of that excellent moment in Torchwood, Children of Men, when the minister for education says (in a very sinister context) “If we can’t identify the worst performing 10% of our children then what good are league tables anyway?”
Patricia @64: “She never seemed to me to understand that education is not just about academic achievement.”
Umm,…seems to me that you just spent the previous 3 paragraphs explaining how parents are obsessed with academic achievement. Then you criticise Julia for focussing on it too.
The idealists amongst us can chat about other social aspects, to create the well-rounded individual, over our lattes and fine wines. Perhaps most people regard those things as “nice to haves” rather than “must haves”? As you indeed suggested yourself in paras 1-3?
Incidentally, most private schools seem to put considerable emphasis on providing a “well-rounded” experience to the children in their care. Certainly those that I know anything about do so. These things would not be measured in the Myschool metrics.
Ken @59,
shining a light on the performance of private schools, and getting people to ask whether they are value for money, may not be intent of the exercise. It probably was not.
But it may be an unintended consequence.
LO, without evidence that these selection biases are material your criticisms amount to nit picking. You and others are missing the bigger picture, and that is that some poorly performing schools will not be able to hide behind their students’ socio-economic disadvantage as other schools with the same disadvantage are doing better. We can now ask why is it so. And we can ask why apparently well performing schools aren’t doing all that well
Is this a statistically pure exercise? No. By all means fix up the comparator groups to remove the errors and anomalies. But IMO it is a breakthrough in a debate where nothing new has been said for 40 years.
Not using the simplistic “similar school” comparison they’ve provided.
Even if third parties do more accurate like with like comparisons, my guess that there is so much noise in the data that differences between individual schools with similar demographic profiles will mostly be insignificant, and statistically significant data will usually be the result of cohort factors rather than anything that can be put down to the school.
Finally able to log on and check my son’s school rating and cannot stop laughing. That system, apparently designed most of all to assist parents from socioeconomically disadvantaged backgrounds make better choices, is a joke because it is almost incomprehensible. Think about how poorly educated parents are supposed to make sense of that. I just love the ‘tell me more’ link which takes you a page a stastical analysis. Gillard’s wonks have fucked up big time on this.
I suspect that this web site is of very partial value. Its greatest use is for those poeople who are socially isolated and for the parents of overseas students. It was my observation that mothers had this this whole schools grading thing sorted while their first children were in preschool. The massive interest is more, I suspect, to do with mothers checking whether the parents network perceptions were backed up. At the end of the day kids still have to go to the scholls in their area for a whole lot of practical reasons.
Does anyone actually know what the standardised NAPLAN tests are designed to test?
Only when I chose to know, Fmark.
LO #29
Which is why you get absurdities such as Fairholme College, an ultra-elite private Protestant girls school in Toowoomba, being classed as “statistically similar” to Greenslopes State School in a relatively working class suburb of Brisbane, and Kotara High School in Newcastle.
BilB @ 70 – And for parents who move interstate who often also have some choice about where they will live. Agreed that the mothers/parents groups often have a good idea already, and it was interesting to see that at least from a quick check the results on the website have aligned pretty well with what the parents with older children we’ve talked to have said.
Sam @ 67 – the other question that can be asked if you believe the comparisons are accurate is if extra funding for many public schools is worth it because the extra money doesn’t seem to help the private schools much? Perhaps the government would be getting returns instead by assisting children and their parents outside of the school environment.
Fairholme, by the way, is a boarding school whose inmates are sent there by well-heeled pastoralist parents from all over rural Queensland.
Whilst on the subject of boarding schools, Hurlstone Agricultural High School is deemed to be “statistically similar” to a swathe of schools in working class suburbs. It would appear (and if someone up the thread has addressed this, please forgive me) that in cases such as Fairholme and Hurlstone, the “statistical similarity” is with schools in demographically similar suburbs/towns to the suburb/town in which the boarding school is located, when surely the basis for demographic comparison has to be the demographic characteristics of the far-flung families who send their kids to the boarding school.
Elise, sometimes late at night we miss a step in the logic of our argument. Never mind I love the idea of myself as a latte and fine wine sipping idealist. I confess to the lattes (soy at that!) but cask red is my occasional tipple. I’d like to think I was more realist than idealist. I re-read your comment hoping you’d accused me of being an elitist! Then I could have really got my dander up!
Let me clarify that I have no problems with the publishing of this information, particularly since it does confirm and will encourage in the public mind my own strongly held opinion that our state school system is doing its job pretty well. It might also stem the drift to private schools by parents who can do better things with their money.
As a near street urchin who was an 11 Plus beneficiary of the UK 1944 Education Act I know the benefits of scholarship. That’s not to say I would not have done as well had I attended the later more egalitarian secondary modern comprehensive which incorporated my elitist grammar school against the strident protests of many parents. They, like Australian parents of today, were too often dazzled by the “results” published then in terms of university places achieved by the few.
Nowadays education systems everywhere are so much broader and post-secondary education comprises more than universities. CAEs and technical college campuses abound in all major centres and even those who don’t manage to finish Year 10 can later get advice on how to squeeze into vocational courses which give them that training and piece of paper they now know they need. My particular regret for them is that they missed out and were unhappy during their teenage years when others, already well endowed with wit and the general wherewithal to do well were given more than the lion’s share of the school’s resources.
My comment about Julia Gillard stands. She’s a great political fighter but she doesn’t seem to understand that most teachers, like me, are not elitist and know that academic achievement is not the only benchmark by which to judge a school. Why choose this particular issue to champion when there are others in government better equipped and well able to resist or even reconcile with the teachers’ unions? They’re a mighty stubborn bunch, well organised and won’t take well to defeat. Let’s hope that for Labor’s sake there’ll be a reconciliation, or at least a truce, before this looming election.
Patricia, you’ve blown your cover. Behind your online facade as a school teacher of advancing age, you’ree really a 19 year old University student in shared digs.
Paul, it appears that it’s done by examining the demographic characteristics of the localities where the students come from.
This means that for each student, they look at the locality the student comes from, and take the average demographics of that locality. In other words, each student is assumed to come from an average household from their home town (or suburb) for the purposes of calculating the socio-economic score of the school.
For elite private schools that take the children of the landed gentry, , or elite government selective schools, that assumption is, frankly, nonsense.
The further point, as made upthread, is this is very similar nonsense to that used in the funding formula devised by the Howard government – and maintained by the Rudd government – for federal funding of private schools.
Thanks Robert.
Dammit, now property valuations, and therefore council rates, in the Brisbane State High catchment will be bumped further skywards: why burn $100,000 or so, with no capital gain potential, per little darling, when you can get a better result (than at only 4 other schools in queensland, and 3 of them don’t take boys) for free by securing access to a letterbox in the right postcode? You can always turn the address over after the 5 years or so if it turns out you don’t like being a stroll away from the CBD, art galleries, state and council libraries, museum, state theatre and concert halls, southbank beach, barista boulevard, tafes, griffith and uq unis, 191 bus etc etc.
I better start sub-dividing those spare rooms to pay for the rates increase. Are you allowed to auction street addresses on ebay?
Patricia @76: “I confess to the lattes (soy at that!) but cask red is my occasional tipple. I’d like to think I was more realist than idealist.”
I’m fond of weak espressos, and have drifted towards SSB, lighter reds and roses, these days. Totally relate to your second sentence there.
More particularly, thinking about teachers unions, I suspect most teachers think these metrics will be used as a stick to beat them with. Fear of the unknown.
Given the shortage of teachers, I doubt any would lose their jobs unless they were really, truely hopeless. One might surmise that most teachers are dedicated to their work, as it is not the best paid profession, with limited opportunities for advancement. Those hopeless ones, who obviously aren’t suited to teaching, might have been weeded out by the previous system anyway.
More particularly, public metrics should create the potential driver for better facilities to be provided for schools that are providing a poor learning environment relative to their peers.
It may also lift everybody’s game in the schooling system, which is not a bad thing, considering that Australia has been falling behind in the international metrics for education. We really do have to lift this area for the future, for all Australian children. Aiming for outstanding education of children is a fundamentally valuable concept. Totally agree with Rudd and Gillard on this.
It may be uncomfortable in the short term, but the end objective is undeniably desirable. Hopefully the teachers union doesn’t lose sight of this and get carried away with fear, losing sight of their own role in driving change?
This is a case of much ado about nothing. Schools are ranked every year in HSC results and nobody batts and eyelid. Why are we so afraid of transparency? In the U.S the idea that this information should be kept secret to protect the poor little below average schools would be dismissed as ridiculous because they have a much stronger culture of public accountability. I’m fine with the concept.
The only criticism I have is that schools that select for academic performance are compared with those that don’t, which really is an unfair comparison. It would also be good if you could sort by State, and public/private so that you can gauge performance of schools in your local area. I don’t give a toss how my local school compares with one in Queensland since I have no idea of the characteristics of that area.
Paul Norton @ 77 – if only!
On second thoughts, though, if I had the choice I’d be a thirty-nine year old. Old enough to know the score and still young enough to better it.
I am spending and will spend vast sums educating my offspring (I don’t get a choice – Mrs Razor is a teacher and she has spoken). Although academic performance is important, I also want them to particpate in the broader range of activities that are offered by private schooling. The local government schools don’t provide a wide variety of sporting and cultural activities that a private school does. One very difficult thing to measure that you get from a private school is esprit de corps – school spirit. The pride in being part of something that has a long and proud history and traditions. For the majority of students it remains with you for life.
I have no doubt that I could buy a house in one of the areas with the best governemnt schools to get a good academic outcome and package together a range of after-school activites from various providers to match the activity rates of private schools, but you still would miss out on the esprit de corps.
Danny @ 80 – I know someone who uses the Grand parents address as the family address to get the kids into the government school of their choice.
ICSEA is bizarre. The technical paper shows it is grounded upon a non-sequitur. See the first page – “Unfortunately no such measures of starting abilities are currently available nationally, so instead, attention focused on finding proxy measures that are highly correlated with student performance.”
Performance does not indicate abilities; it never has and probably never will. Proxy measures that are inevitably highly influenced by the social standing of your parents will give even less insight.
This is Ayn Rand applied to education – it pretends the “achievements” of parents are a sound guide to the ability of their children to “achieve”. Talk about flicking the switch to vaudeville.
Lun @86: “Performance does not indicate abilities; it never has and probably never will.”
Yep. And ability does not guarantee performance or success or social standing.
Has anyone ever see a study where they have found a stong, reliable correlation to any metric.
I’ve just read the ICSEA Technical Paper. In essence the ICSEA was developed specifically to predict a schools known NAPLAN results. The census data was mathematically fiddled to generate a formula which best predicts the known NAPLAN results.
Clearly (and understandably) the ICSEA scores are far from perfect NAPLAN predictors. If ICSEA scores were perfect predictors of NAPLAN results then all schools would be white in all their SIMilar boxes. As a consequence, (it seems to me) all the red, pink, light green and green colours in the SIMilar boxes indicate errors in the ability of ICSEA scores to predict NAPLAN results. Nothing to do with school performance at all!
SamD @88: “If ICSEA scores were perfect predictors of NAPLAN results then all schools would be white in all their SIMilar boxes.”
So, does that mean that there should be NO influence from the school itself?
Of course indivudal schools have an influence on NAPLAN results. I just can’t see how the ICSEA methodology used to compare similar schools targets any influence a school may have on the NAPLAN results.
SamD @90, I may not understand the system either, but perhaps they are just controlling for some of the variables with ICSEA?
Say variable A and B affect outcome C. So you gather together everything with the same A. If A were the main influence, then you could predict outcome C. All white in similar boxes.
However, say B does an influence on outcomes. Then C could not be predicted by just controlling for A. The boxes should be different colours according to the influence of variable B.
Ideally you would control as accurately as possible for variable A, so that you got a “pure reading” on the effect of variable B. I think this is what a lot of teachers are saying, with some justification.
However, there is less scope for fiddling with the outcome C, which can either be compared imprecisely with similar A’s or compared against all other C’s. That is, you can compare imprecisely with other “similar” schools, or without filtering against all other schools. The tables give you that choice.
Hope that makes sense?
The inability to develop ICSEA scores that perfectly accurately predict NAPLAN results would be due to numerous factors. No doubt the school’s influence would be one, however there would be numerous others that have nothing to do with the school’s teachers and educational programs. My issue is that the comparison of similar schools implies that it is the influence of individual schools that makes the difference when I find NO evidence to support this assumption.
SamD @92: “I find NO evidence to support this assumption.”
You mean quality of education doesn’t affect results?
Would additional tutoring be a similar waste of time?
Or is it specifically our schooling system?
If quality of schooling doesn’t matter, they may as well wag school, and sit under a tree in the back yard or play with the dogs? As a lot of aboriginal kids are doing at the moment? Same outcome – I think not.
I’m not being rude here, but simply illustrating the point. If something is deemed to have NO influence, then test what happens for not doing it at all. It should yield similar results.
Elise @93 you misunderstand my issue. Of course quality of schooling affects results. My point is that there is no evidence to confirm that the differences between similar schools on the MySchool website is substantially due to differences in the quality of education.
One obvious example: When grouping similar schools selective schools are included with comprehensive schools. The students are from similar socio-economic backgrounds (according to ICSEA scores). However the selective schools achieved vastly better NAPLAN results. In this case one would assume the differences in SIM shown on the website has little to do with the quality of education and much more to do with the quality of the initial student inputs.
One would presume that selecting brighter students also occurs in a variety of less obvious ways – for instance, one would presume parents who value education are more likely to spend a larger proportion of their income on private education regardless of their socio-economic statistics.
My initial point is that inaccuracies in the methodology used to create the ICSEA scores is another significant factor which would cause differences between schools the MySchool website claims are similar.
The big problem is not so much that it shows so called good schools and bad schools. We already know them. The problem is that is restricts education to boring literacy and numeracy tests. I am a teacher who has administered these tests. they are boring and kids find them boring. They have some skills on which competent students should know but if we only had to know that then we would be very boring people. This site is lacking because it can never measure the kids teachers save from going off the rails, the child who was from a disgusting homw who a teacher turns around or the child who learns to love school through art or through manula arts. Gillard does not care though. She is just another careerist with no idea and a big ego.
SamD @94: “The students are from similar socio-economic backgrounds (according to ICSEA scores). However the selective schools achieved vastly better NAPLAN results. In this case one would assume the differences in SIM shown on the website has little to do with the quality of education and much more to do with the quality of the initial student inputs.”
Yes. Agreed. I think we are on the same page now.
If you start with promising material, your results are likely to be better, even if your processing methods are similar.
To give a really cynical example, the top business schools use the GMAT test (amongst other things in a detailed screening) to select their candidates. They publish GMAT thresholds for the different schools. (GMAT is a test for deductive reasoning, amongst other things.) These schools then publish the 3 year out salary increases for their graduates, to show how successful they are. They also ask for your current salary in the entrance questionnaire, and they train you how to negotiate a high salary.
Sooo, if a business school produces graduates with high salaries, was it because:
- they were bright (high GMAT scores)
- they got brilliant training at the business school
- they were already fairly successful and were selected on already proven ability to achieve high salaries
- they got specialist training in how to extract the MAX, like Trujillo
The overt story is the first two items (clever people and elite training), but I reckon the covert story of the last two has quite a lot to do with it. But that would ruin the mystique of the business schools.
Back to MySchools, selective schools could be expected to achieve better academic results if they already screen on academic skills. Provided that they provide an adequate educational environment to capitalise on this potential.
Even so, perhaps selection may not be a bad thing? Not all kids are cut out for high levels of academic achievement. There are many other ways to live a successful life, as creative people (e.g. musicians, artists, writers), entertainers (e.g. actors, comedians), sports professionals (e.g. footy, golf, cricket), etc, etc.
Putting these kids in with the ones with academic skills may only make their lives miserable, because the system at that stage does not value what they have to offer? Humans are comparative creatures, and their happiness or misery depends on how their lot compares with others around them.
It is a difficult balancing act I guess, but ideally we want a system which is flexible enough to maximise the different potential of all children. Nevertheless it need to give them all a solid enough grounding to function and have future choices in modern society.
Perhaps MySchools would be better viewed as “qualitative data” – a baseline indicator of a good standard of education, but not the be-all and end-all?
*sigh* – I worked at ACER a few years when it was contracted to assist in developing the first NAPLAN, and assisted in writing numeracy questions.
1. NAPLAN is a diagnostic instrument not a test. Like all instruments it has limitations. The instrument designers are well aware of the limitations.
2. NAPLAN was developed (at least in part) to replace the collection of different diagnostic instruments that the states/territories used. This better allows results to be compared across states/territories.
3. NAPLAN looks (from memory) at core skills. It’s very good to know if a kid is particularly poor at core numeracy or literacy, especially given their importance to kids’ future prospects. Some kids are a long way down the tail.
4. Which schools are taken to be “statistically similar” doesn’t seem to be a key point.
5. Whether individual “improvement” should be published is an interesting question.
Thanks Sacha @97!
That was very helpful information.
Sacha, the whole point of the My School website is that it compares schools NAPLAN scores against a cohort of “statistically similar” schools.
In that context, whether “statistically similar” schools are actually similar enough to make sensible comparisons is a key point.
If the point of the exercise is measuring individual student achievement, it of course doesn’t matter a whit.
For this interested in the website technical details, it is written in .Net (Microsoft platform) and running on Windows Server 2003.
There seems to be a trend in government towards using expensive proprietary software for government website, another exampble being myki.com.au in Victoria for the shambles that is supposed to be a new public transport ticket system. [link]
Neither seem to handle high traffic well either.
It has become clear, since last night, that Julia Gillard only wants a school league table to help her choose which children will be fed to the 456
Heh, Ewe2 made a similar joke on Mercurius’ thread Joe2.
So maybe they should stick to publishing the raw averages (and the medians, that would be useful) and then have a general page discussion about the myriad of reasons why those averages may not be comparable across schools. They seem to have shot themselves in the foot by attempting to quantify this in some way.
There are so many threads on this topic here that this may have already been mentioned but public schools tend to have higher proportions of integrated students with intellectual disabilites. When I was enrolling my older son, the private schools I spoke to positively discouraged enrollment by saying that there was less funding available to them to support such students.
hey I made that joke here first!
I’m Spartacus!
10 points from Slytherin for moaning SG.
Drats! I thought it was a first. I fell about laughing when I saw that bit in Torchwood last night. I love to think Julia was watching it.
I think it’s worth remembering what “statistically similar” means in this context. ICSEA combines 15 different variables (Admittedly not all completely independent) to produce a single number. It would be possible for two schools to have completely different numbers for every variable – thus representing completely different demographics – and yet happen to produce final scores that are similar. The designers of ICSEA would say that the two populations may not be “similar” in terms of demographic composition but are similar in the level of educational advantage their students come into school with.
Of course a lot seems to hang on this claim.
I get that Martin, but the idea that Tallangatta Secondary College and MacRob are remotely comparable suggests that their similarity measure is hogwash.
Furthermore, I’m unconvinced that you can usefully produce a one-dimensional index of educational advantage and use it to compare schools in an interesting way. I’m not an expert in this kind of thing, but I’m innately dubious of the proposition.
“Furthermore, I’m unconvinced that you can usefully produce a one-dimensional index of educational advantage and use it to compare schools in an interesting way. I’m not an expert in this kind of thing, but I’m innately dubious of the proposition.”
In principle I think it is possible to have a single index that is comparable, but the authorities would need access to data that was not available in this exercise. You can think of the problems of the index as a combination of omitted variable and selection biases that are not adequately controlled for.
I absolutely agree (with equal or less level of expertise
). That’s largely the point underlying my post.
I’m willing to accept that the ICSEA scores are able to predict a school’s NAPLAN results with some accuracy. However, the MySchool site then compares their prediction to the school’s actual NAPLAN results. The site then implies that somehow the school’s performance is better or worse than similar schools simply because its actual NAPLAN results don’t precisely match their prediction.
Logically this is surely nonsensical!
Another issue pertinent to Rob’s initial example is that, dimensional compression notwithstanding, selective schools like MacRob are going to be less well measured by ICSEA because their actually enrolled students are likely to be less representative of their census districts if for no other reason that they will be generally a smaller sample of those districts. (And of course there are other reasons we might suspect.)
MartinB how does the ICSEA rank handle the one third of MacRob students from overseas who pay full fees
I would presume that they take data from their residential address in Australia.
Heh. My kid’s school, a little 250 kid inner-suburb Brisbane primary school, is statistically similar to only one other school in Qld and is in the same group as Fort Street High School and Northcote High School.
d
ooh. I just looked at Brisbane State High, the biggest and fanciest State school in Qld, and it’s also statistically similar to Fort Street and Northcote, but NOT similar to my kid’s primary school.
d
I did notice BSHS and Northcote were on the same page
Meanwhile, on the politics of MySchool, Julia Gillard continues to be applauded by people who will never vote Labor anyway.
“Meanwhile, on the politics of MySchool, Julia Gillard continues to be applauded by people who will never vote Labor anyway.”
Yes, intelligent words from Miranda Devine — Teach for America must be the way to go, and when you need evidence, you should always ask the organization you are evaluating to provide it (or look at their web site, even easier). You always want unbiased independent evidence, after all.
If she’d bothered to do a better search, she would have found that at best tiny positive effects from these sorts of programs, even when you include teachers as inexperienced as the Teach-for-America ones into your control group (e.g., this publication). This doesn’t even take into account all of the other problems the program creates, like the 2 year turnover, dissatisfaction amongst the TFA teachers etc. .
The other strange thing about it is that she obviously doesn’t read the paper she comments for. Had she done that, she would have found out that negative results are found from these programs in some cases also (look e.g. here)
Why do we need Teach For Australia?
For the past 20 years each year there more 8000 trained teachers looking for work than there are positions in Victoria. When principals say they can’t find teachers they mean they can’t find slender 25 year old Melbourne Uni graduates who studied at a western suburbs high school. After 18 months hunting for work I have developed a league table of where not to study for a Dip Ed and where to study – if you want to teach.
Billie, I dunno what it’s like now, but my understanding is that there is generally no shortage of English and humanities teachers, but a perpetual shortage of mathematics, science, and foreign language teachers in high schools.
Most principals look for young male maths / science teachers.
Since Victorian school find their own teachers, decentralised staffing, the are very vague numbers for numbers of graduates, number of experienced tteachers moving in, vacancies.
Without accurate statistics there are inaccurate numbers and poor policy with no ability to plan for the future systematically and improve the system
Until 2005 all vacancies longer than 2 weeks were published in the Teaching Gazette. Today only vacancies longer than 6 months are advertised. These vacancies are advertised on the Education Department online website. It’s not a copy of the very effective TAFE staffing website, its an American Peoplesoft mess.
It seems the predictions that the publication of Naplan results will cause schools to shift their focus to “teach to the test” (already noted in the UK) are correct.