« profile & posts archive

This author has written 591 posts for Larvatus Prodeo.

Return to: Homepage | Blog Index

131 responses to “Did the East Anglia hackers score an own goal?”

  1. david_h

    Maybe its the hour but I found points 5 & 6 amusing.

    5. Instead of holding a press conference, climate scientists like to keep the fisticuffs in the confines of juried journals.
    6. Climate “skeptics” are full of themselves, and probably wrongly accuse climate scientists of fixing data.

  2. tssk

    Not an own goal. The link made a good case but for most people it will be TL:DR

  3. still@downfall

    Sorry Brian, saw no home goal in this link, just more fodder for this bun fight to continue.
    In the meantime KRudd’s CPRS bill will be voted this week and it’s not just right wing hicks who oppose ETS type legislation, here in smh,This ETS-lite deserves to be rejected, Friends of the Earth are quoted.

    Australia (and other countries) would be better off with no ETS. Two recent reports – The Brave New World of Carbon Trading by Australian ecological economist, Clive L. Spash, and A Dangerous Obsession by Friends of the Earth – spell out in detail why the attempt to deal with global warming by setting up ETS schemes have already failed, why they will continue to fail and also why governments, in thrall to financial interests, continue to persevere with them.

    Both papers argue that the most worrying aspect of the ETS debate is the way in which an economic theory with little relationship to political reality is being used to justify the creation of new financial instruments and a new commodity market.

    According to Friends of the Earth, the reliance of carbon trading on offsetting and the difficulty in regulating it creates the risk ”that it could become another speculative bubble which eventually bursts – much like the subprime mortgage crisis, but this time with implications for global climate mitigation as well as the economy”.

    The GFC was a global crisis causing international banks to seek massive hand-outs, yet the ETS proposals will place a $3 trillion market in the hands of the same people and organisations.

  4. Lefty E

    7. Fraud in climate science may occur, but generally on the side of those who argue against warming or who advocate inaction as a response.

    Well I wont hold back: There’s your ‘gotcha’. The ones alleged to be cooking data were actually the “skeptics”.

    Is anyone that surprised?

  5. tssk

    I’m with the skeptics. Or rather I believe in climate change but I’m hoping like hell now that the skeptics are right and all us ‘warmanistas’ are wrong wrong wrong. As I’ve said before the only action now is for those of us that believe to make the sacrifices neccessary.

  6. Lachlan O'Dea

    All the article seems to be saying that the CRU people thought a paper was fraudulent because it said their modelling didn’t agree with observation. Maybe they are right about that, but either way it’s hardly surprising to find them expressing that opinion. Am I missing something? (genuine question).

    I was surprised at how quickly the crowd here has dismissed this leak. It could be true that nothing unethical or deceitful was going on are CRU, but I think this leak raises many legitimate questions that are worthy of investigation.

    A more recent discovery is programmer comments in the modelling code, which may indicate artificial adjustment to make model results match up with observation. Details. Yes, I just linked to an “enemy” site, so there goes whatever little credibility I had around here :-) .

  7. Andy

    For those who mindlessly repeat the mantra that CO2 drives temperatures and cite the ice cores as proof, here’s the link to the study: Timing of Atmospheric CO2 and Antarctic Temperature Changes Across Termination III. And here’s a link to RealScience page that purports to explain the anomaly: Real Science.

    I’ll summarize for you.

    – In the ice cores CO2 rises 800 years (+/- 200) AFTER temperatures rise.

    RealScience.org provides a plausible explanation but the fact is, they don’t know. Their explanation is nothing more than speculation. CO2 does increase temperatures but by how much is unknown. The relationship is not well understood. For all the climate scientists know the recent rise in CO2 could be in response to the Medieval Warm Period (MWP). You know, the one that had to be eliminated by M. Mann.

    I have some advice for the researchers in this field. If you don’t know, don’t be so weak that you don’t have the courage to say you don’t know.

  8. Paul Burns

    Like, this is going to get massive publicity on the ABC and Herald-Sun? Not. ABC2 Morning TV twice broadcast an outraged Bolt = as if they weren’t sure Australia had got the message yet.

  9. Brian

    First up, please note that I’ve taken no position on the subject of the link. I thought the author painted climate scientists with a somewhat rosy hue, but agree that cases of outright venality are hard to find, if there are indeed any.

    OTOH the skeptic side routinely cherry pick, distort and handle graphs in an unscientific way. I recall posts at Open Mind where Tamino has the mathematical skill to deconstruct graphs served up to prove that AGWs nonsense. He regularly finds stuff that can only be knowingly dishonest.

    I gather he had an accident which limited his posting, bit he has risen to the occasion with this post on the hacking incident. Go read it, it’s not long, but if you are too lazy or time-challenged here’s the guts of it:

    The denialosphere has trumpeted the contents as proof of the fraudulent behavior of climate scientists, especially Phil Jones at CRU. But what’s most remarkable is that even the bits pointed to as a “smoking gun” really don’t support that idea. There are certainly phrases which seem incriminating when taken out of context — but when put into context are nothing of the kind.

    Continuing to suggest that climate scientists generally, and Phil Jones specifically, are engaged in a conspiracy to deceive the world about global warming, when there turns out to be no real evidence of it in 10 years of personal communications (only words that can be twisted when taken out of context), demonstrates the idiocy of those who stand by that suggestion. If anything, the messages prove that there is not any conspiracy, and the scientists at CRU did not fudge data or engage in deceptive practices to push their “agenda.”

    Certainly the emails contain some unkind words about certain people. I’ve said unkind things about some of them myself (here on this blog for all to see). In my opinion, the unkind words were earned by the loathesome recipients.

    I value Tamino’s competence and integrity in these matters.

  10. Brian

    Andy @ 7, you’ve chosen a 2003 paper and a 2004 response. This issue is an old chestnut. Also I’ve asked you not to turn the thread into an argument about climate science as such, or we’ll go down the tiresome track of the previous thread.

  11. Paul Norton

    In terms of political consequence, the East Anglia hackers will certainly have scored an own goal if they have managed to nudge enough Liberal MPs into the denialist camp to scupper Turnbull’s deal with Labor on the CPRS, thereby sinking the entire scheme and dealing the Greens into the climate policy game when the Rudd government has to revisit the issue after the next election. We’ll know soon enough.

  12. Brian

    Lachlan @ 6, there is a good section on modelling in this book. My maths is almost non-existent, but I gather that in constructing models what you do is design a model that as near as possible matches historical observations. Otherwise it would be useless. It’s how you do it that matters. You can tweak it so that it matches the historical data at every point but end up with a model that is useless for predictive power or in gaining insights unless you incorporate scientific principles and appropriate relationships between the elements.

    Morgan and McCrystal reckon that the richest contribution of models is that “they throw the nature and extent of our knowledge of key drivers and their uncertainties into sharp relief.”

    That’s a value quite apart from any predictive power.

  13. Lefty E

    I’d like know at what point of “unsettled” various commentators here think we shouldnt act to avert or mitigate dangerous climate change, given the potential consequences

    99% settled, 90%? 80%? Lets say you have a 80% risk of a fatal illness if you continue smoking. The diagnosis is technically “unsettled”. Are you feeling relaxed? Wondering what all the hoohaa is about, when there’s no consensus on your forthcoming untimely death? Cranking up another ciggy, with a song in your heart?

    It doesn’t really mean much, does it? Debates are really at the margins now. At this point we still have to act.

  14. wozza2

    Brian, as far as I can see, your link takes one paper from the vast range of hacked material and declares victory on the grounds that that paper is fraudulent and is written by deniers. And on that basis (and even the claim of fraudulence is of course just that, merely a claim made by an AGW adherent) all the rest of the material showing some highly suspicious behaviour, to put it no higher, on the part of the AGW-ists can now be ignored?

    I would have to say that the debate here to date was brilliantly summed up by Yobbo @192 on the previous thread. There is a remarkable amount of clutching at straws going on here.

    Mercurius takes the straw-clutching gold, though, when, in relation to the unremarkable statement “Genuine scientists don’t adjust data to reach their goal”, he ripostes brilliantly “Understanding what scientists do fail.”

    Of course, the belief that it is OK for scientists to fake data to back up a pre-determined conclusion when the tenets of AGW are at risk might explain quite a bit about the previous thread.

  15. Brian

    wozza2, I asked a question, I didn’t declare victory. And FWIW I thought Yobbo’s comment was one of the most pointless in the whole thread.

  16. Nickws

    The smoking gun e-mails show correct and careful behavior by the scientists who contributed to the IPCC report, but unethical behavior by the critics whose backers, we might assume, stole the e-mails in the first place, and published them without understanding the depth of moral character demonstrated by most scientists in the conduct of their professions.

    Ah, thank god for that. Now we just have to wait for the media narrative to rectify itself presently.

    Though when I saw Andrew Bolt calling the assaulted scientists a ‘cabal’ I thought to myself, “Nothing says winner like big noting yourself as an opponent of criminal conspiracy.”

  17. Ootz

    Lachlan @6

    You have not linked to an ‘enemy’ site rather than a site which is representing a view from an individual which is associated with the Heartland Institute. I do not need leaked emails to understand where these characters are coming from. The Institute holds a view that tobacco smoking does not affect your health. I suggest you ask your family doctor what his opinion is on that. You very likely find he is an alarmist in that regard too. So rather than label it an enemy site I would prefer the term unreliable site. I’d go to Monash or other reputable and long established universities to form my opinion about a scientific question of great importance. Leaked emails make great news for tabloid newspaper circulation a bit like p3 tits and give those clutching at straw something to temporarily hang on to. Let me assure you, the climate does not do leaked emails and my bet is it will carry on in its projected path regardless.

  18. Brian

    And amongst it all I don’t recall any evidence at all of climate scientists faking data. Plenty of assertions, but no evidence.

  19. Zarquon

    “Genuine scientists don’t adjust data to reach their goal”

    The problem is, the denialists pretend that this means that scientists never adjust data, and take any discussion of doing this as evidence of wrongdoing.

    In fact, raw data is calibrated, smoothed, recalibrated and any number of techniques are used and discussed as ways to present the data. As long as those techniques are justified in the peer-reviewed literature, there’s no problem.

    The idea that data analysis is the same as data faking is the straw that breaks the camel’s back as far as the intellectual honesty of the denialists is concerned.

  20. Mercurius

    Of course, the belief that it is OK for scientists to fake data to back up a pre-determined conclusion when the tenets of AGW are at risk might explain quite a bit about the previous thread.

    Nobody believes its OK for scientists to fake data to back up a pre-determined conclusion, least of all scientists. Scientists caught doing it are drummed out of the profession and all their findings are overturned and swiftly forgotten.

    That you believe these CRU emails are evidence of scientific fraud on the part of the climate scientists means that you too are an ‘understanding what scientists do fail’.

    Adjusting data and faking data are not the same thing. Adjusting data is a normal part of the workaday world for practitioners of science (and medicine, and, and, and…) Faking data is what charlatans do. If you wish to maintain an obtuse lack of awareness of this distinction, then you are wilfully ignorant.

  21. Nickws

    And amongst it all I don’t recall any evidence at all of climate scientists faking data. Plenty of assertions, but no evidence.

    The best quick devil’s advocate analysis I’ve read is
    Nate Silver’s ‘I Read Through 160,000,000 Bytes of Hacked Files And All I Got Was This Lousy E-Mail’.

    To wit, “What you have is a scientist, Dr. J____, talking candidly about sexing up a graph to make his conclusions more persuasive. This is not a good thing thing to do — I’d go so far as to call it unethical — and J____ deserves some of the loss of face that he will suffer…
    But let’s be clear: J____ is talking to his colleagues about making a prettier picture out of his data, and not about manipulating the data itself.”

    I don’t have to be a concern troll prattling on about `warmanistas’ to see that there is no substance to fraud here, no creation of data, just proof that one scientist has committed the crime of spinning (I’m going out on a limb and trusting Silver’s reporting).

  22. Fran Barlow

    Quite right Merc and Zarq …

    Mind you, I really hate the fraud associated with seasonally-adjusting unemployment numbers. Who are these boffins trying to fool? ;-)

  23. Sam

    Shadow cabinet has approved the deal.

    The rent seekers have won a great victory.

    The denialists in the opposition will be enraged, especially now that emails have proved that the climate change is a communist fraud, yada yada yada.

    Let the games begin.

  24. Paul Norton

    Keep your eyes peeled for a white bear with brown spots.

  25. anthony nolan

    Anyone located Godwin Gretch working this scene?

  26. Paul Burns

    Why not a brown bear with white spots?

  27. Paul Norton

    Paul B #26, no reason why not. For that matter, why not a roan bear?

  28. anthony nolan

    BTW and thanks to Still @ above for this press release:

    “Both papers argue that the most worrying aspect of the ETS debate is the way in which an economic theory with little relationship to political reality is being used to justify the creation of new financial instruments and a new commodity market.”

    The argument that market solutions are inadequate to the task has been well made over the last two decades. A CPRS will not adequately reduce carbon production because of the lead time that the so called ‘market signals’ take to produce and effect. If you think that commodifying a resource (like the global atmosphere) will produce desirable outcomes then you haven’t been paying attention to what has happened here (Murray-Darling) and elsewhere with water trading.

    Tax carbon production massively and urgently, regulate production and consumption of resources, move the economy to an emergency setting so that food and water security become priority issues.

  29. Andy

    Brian @ 10

    Fair enough. I won’t bother to point out that in #75 on the previous thread that is exactly what you were doing. I also won’t ask for citations (real academic citations, not blog bs) from you that dispute what I said. I also won’t point out that the controversy surrounding the leaks IS about the science.

    Don’t bother banning me. I’m done here.

  30. tssk

    I’d like to point out another scandal in the scientific community.

    See this link for the truth.

    http://carbonfixated.com/newtongate-the-final-nail-in-the-coffin-of-renaissance-and-enlightenment-thinking/

    Short version. Letters from Isaac Newton’s personal collection show that he conspired to avoid public scrutiny, insulted dissenting scientists, manipulated and suppressed evidence, knowingly published scientific fraud, abused the peer review system and insulted critics.

    Therefore anything he wrote on gravity, optics and calculus is now in disrepute.

    Devastating!

  31. Steve 1

    In politics and the social sciences, what we say and do does need to coincide with what we actually think and believe otherwise there will be ‘gotcha’ moments. In the physical sciences, what people think or believe is basically irrelevant because it is all about the published data that is open to peer review. One scientist will print results from an experiment, another will try to reconstruct the same experiment to test whether the results are valid. Scietific reasoning is not about proving a proposition is valid, it is about proving that a proposition is not invalid. as I understand it, those who have science degrees who are in the climate change denier’s camp, do not publish their data in a format or journal that allows them to be subject to peer review, so while they may be ‘scientist’, they do not use the scientific method to validate their position. The fact that most reasonably thinking people would consider the climate change deniers gormless gits who would probably be denying gravity in Newtons day, it is what you produce for peer review that is important. The fact the deniers have to resort to stealing private correpondence to bolster their case shows how thin the ice they are skating on.

  32. Mole

    Ok heres a fairly beefy post from another site dealing with some of the data codeing.

    If its not fraud Im pretty sure you be on safe ground calling it an enormous shambles.

    http://www.devilskitchen.me.uk/2009/11/data-horribilis-harryreadmetxt-file.html

    Apparently the text is from coders notes as they fix problems in the database for the information.

    a couple of sample (sorry for the cut and paste, but Im limiting it to a couple of standouts)

    “..So.. we don’t have the coefficients files (just .eps plots of something). But what are all those monthly files? DON’T KNOW, UNDOCUMENTED. Wherever I look, there are data files, no info about what they are other than their names. And that’s useless.. take the above example, the filenames in the _mon and _ann directories are identical, but the contents are not. And the only difference is that one directory is apparently ‘monthly’ and the other ‘annual’ – yet both contain monthly files…”

    “..ARGH. Just went back to check on synthetic production. Apparently – I have no memory of this at all – we’re not doing observed rain days! It’s all synthetic from 1990 onwards. So I’m going to need conditionals in the update program to handle that. And separate gridding before 1989. And what TF happens to station counts?

    OH **** THIS. It’s Sunday evening, I’ve worked all weekend, and just when I thought it was done I’m hitting yet another problem that’s based on the hopeless state of our databases. There is no uniform data integrity, it’s just a catalogue of issues that continues to grow as they’re found…”

    Thats the sort of info being paraded as “best science” and “90% certainty”?
    If thats genuine coders commentry and not faked it does show that the warming apparently shown in the models is well within theur own margins of error.

    Thats just insane.

  33. Don Wigan

    “Mind you, I really hate the fraud associated with seasonally-adjusting unemployment numbers. ”

    Haven’t seen anything recent on that, Fran, but back in the very old days of the CES that was a favourite ploy of the politicians. Each quarter the CES would release two sets of unemployment figures: the actual unemployed, and the seasonally adjusted figures.

    In the fun days of Billy McMahon’s government, Billy or his Minister would base his comments on whatever set showed the lower unemployed. That often meant changing from actual to seasonally adjusted from quarter to quarter and vice versa, but hardly anybody picked up on the inconsistency.

    Sorry if off-topic, Brian, although I think we’re seeing something of the same trend now with selective trawling of the emails.

  34. Zarquon

    Why do one coder’s problems with his dataset imply anything about multiple, independent lines of evidence for global warming? The Hadley Centre is just one of many places, not the whole story. FFS the denialists aren’t just leaping to conclusions, they’re taking ICBMs.

  35. Brian

    Andy, @ 29, I thought about the issue of allowing argument about AGW on the other thread and decided to allow it. But if we do it again here we are going repeat the same old stuff that was argued on the previous thread, which I did find tiresome.

    As you go out of the door, I might just mention that at the bottom of the 2004 RealClimate post you linked to there was a link to a 2007 post. It explains in considerable detail what scientists knew and didn’t know in 2007. I daresay the game has moved on a bit now. But everything said there supports the common view of AGW.

    Now why didn’t you make us a ware of this link? Were you being deliberately misleading or in a rush and missed it? I don’t know, so I won’t make any judgements about your character and motivations.

    I think we need to be cautious in what judgements we make about others when we don’t know the full context and circumstances.

  36. Mole

    Zarquon

    Thats from the heart of AGW, this bloke isnt a wandering coder hes this chap.http://www.cru.uea.ac.uk/cru/people/#Research%20Staff
    Thats Harry….
    I can scarcely think of anything more relevant than how the data is put together, its sources, and its relyability.

    We are talking about 1/10 of a degree changes and pushing then forward 100 years to estimate tempretures.
    Im not a “leet hacker” but I do know one valuable thing about computors, rubbish in, rubbish out.

  37. mitchell porter

    With Brian’s indulgence I’d like to respond just a little bit to Andy #7.

    “For all the climate scientists know the recent rise in CO2 could be in response to the Medieval Warm Period (MWP).”

    I’m sure that sounds especially outlandish to everyone – here we are actually burning several billion tonnes of fossil fuels every year, how absurd is it to suppose that all that CO2 is disappearing while the observed extra CO2 is coming from somewhere else?! However, you might arrive at such a position if you thought (i) natural sinks have the *capacity* to absorb 100% of human emissions (ii) the atmosphere/sink equilibrium point is determined by some other factor, like temperature.

    But Andy, in that case it would surely make far more sense to say that the extra contemporary CO2 comes from the *contemporary* warm period? The medieval warm period ended around 1300 AD. CO2 started going up after industrialization, centuries later. And I assume the idea is that the temperature increase is bringing CO2 out of the natural sinks.

    “In the ice cores CO2 rises 800 years (+/- 200) AFTER temperatures rise… CO2 does increase temperatures but by how much is unknown. The relationship is not well understood.”

    In fact it has been argued that, thanks to the ice cores, we *do* know how much CO2 warms the Earth, but not exactly how! (The greenhouse effect in itself is straightforward, it’s the feedbacks which amplify it that are not.) See my comments, 74 83 84, especially the link to James Hansen in 83. In fact Hansen states categorically that on account of ice-core data, we know “climate sensitivity” to be 3 degrees +/- 0.5 per CO2 doubling (and he shows a picture of the curve-fitting which produces that value), even though the IPCC gives a rather broader spread of values. It’s one of those curious inconsistencies.

  38. Lefty E

    You know, there’s a good reason why matters strictly relating to preparation of a legal case cannot be subject to discovery by an opposing legal team. Its because people canvass all sort of ideas in confidence, testing theories, shooting the shit, yabbering in a tone more casual than they might otherwise (especially on emails between lawyers on the same team). And if people nick and then adduce this stuff – they get prosecuted, and disbarred if they’re lawyers. And it would *absolutely not* be admissable in a court proceeding.

    Some principles apply to FOI – preparatory policy discussions are rarely released, since there’s – by definition – considered to be NO valid public interest in the tossing ideas about stage.

    Then I reiterate- welcome to academia people – NO ONE releases their primary data to competitors before publications are out, and hardly ever afterward.

    This criminal conspiracy (and I use the term precisely) to release what are effectively the equivalent of privileged internal documents would have the Police up yer bum quicker than you could say boo if this was about a legal case.

    How long have they sat on this too? Nice timing, muddy the waters before copenhagen.

    Shame it turned out to be the skeptics who were alleged to be fudging. LOL!! A classic finish to a great match – look forward to the decider in Denmark. :)

  39. Steve at the Pub

    Hackers score an own goal? No.

    But the authors of the hacked emails sure have!

  40. Katz

    Still waiting for someone to say “Because of the revelations from the hack, I’ve changed my mind.”

    Hasn’t happened? Ho hum. Let’s move on.

  41. Martin B

    “In the ice cores CO2 rises 800 years (+/- 200) AFTER temperatures rise.

    RealScience.org provides a plausible explanation but the fact is, they don’t know.”

    In so far as science ‘knows’ (and of course that is never final) they do know. The role of CO2 as both a feedback (from orbital forcing) and a forcing (of additional warming).

    “Their explanation is nothing more than speculation.”

    Wrong. You may think that we don’t know everything about atmospheric physics and that is true. However Orbital dynamics are *extremely* well understood. These show that the amount of orbital forcing is not enough to explain the temperature record of ice ages.

    “CO2 does increase temperatures but by how much is unknown.”

    By precisely how much, no. However there are good constraints, and they are getting better all the time. That’s the process of science.

    “For all the climate scientists know the recent rise in CO2 could be in response to the Medieval Warm Period (MWP).”

    No. Firstly the MWP, whatever it was, was not a result of continuous orbital forcing. Secondly, and more importantly, isotopic signatures give a pretty good handle on the source of the additional CO2 – fossil fuels.

    “I have some advice for the researchers in this field. If you don’t know, don’t be so weak that you don’t have the courage to say you don’t know.”

    Have you read any of the literature? The real scientists are consistently providing errors, confidence intervals, sources of uncertainty. When was the last time you saw a so-called-skeptic even admit the possibility of error, let alone provide a quantitative analysis of it?

  42. Brian

    I’d like to tell a little tale about how the ani-AGW camp handle scientific data.

    Last month I attended a public presentation by Bob Carter who calls himself a scientist, indeed a climate scientist, with no agenda apart from the furtherance of science. The killer slide he showed was this one showing that further CO2 in the atmosphere will have an absolutely negligible affect on temperature. Hence you could burn as much coal as you like, and in the interests of humanity you should, with no significant effect on the climate.

    This really was a killer graph because as I said the other thread:

    the guts of it is whether the trace gases, CO2, methane etc, cause global warming and secondly whether they do so in sufficient measure to cause a significant signal over the noise of natural variation.

    And where the heat goes.

    I mentioned the problem of the graph on another thread at the time and got this response from climate scientist Roger Jones.

    The computer program was fine. The formula was fine. It’s just that someone had inserted 0.4C as the value for climate sensitivity, the amount of warming for the doubling of CO2, instead of the more usual 3C for short term feedbacks as per IPCC AR4.

    Who had done this?

    Someone in an address to the Lavoisier Society. The basis? None, as far as can be told.

    The graph has been picked up as core anti-AGW evidence in Joanne Nova’s The Skeptics Handbook which was launched at the Heartland Conference, the one Senator Fielding attended to get the good oil on climate change and where Bob Carter gave a plenary address.

    Carter’s report of the conference, linked to by Paul Norton at the time (see comment 1), praised the booklet.

    Now, in meeting Bob Carter I thought he was a helluva nice person and sincere in his beliefs. Perhaps a little uncurious about the provenance of his sources.

    As he was, I thought, in giving an unconditional recommendation of Ian Plimer’s book.

    There’s a saying about people who live in glass houses…

  43. David Irving (no relation)

    Lachlan O’Dea @ 6, as someone who’s written software for a living, I can tell you you’d be surprised at the comments you’ll find in source code:

    “This is borked”

    “You won’t understand this so don’t even try”

    “Warning: ugly hack alert”

    and so on.

  44. Brian

    mitchell @ 37 and MartinB @ 41, that’s good, but from the experience on the other thread, especially James, there is no limit to the number of balls the anti-AGW camp can put into play and if you don’t scurry around batting them back they’ll assume you can’t. And if you do, expect the same ball to appear later as though you’d said nothing.

    I find it tiresome and don’t want to play that game.

    I can’t sit here and moderate the comments out, so it’s probably best to ignore them.

  45. amortiser

    George Monboit has even had to acknowledge there are now serious problems and has called on Phil Jones to resign. He feels so deceived but desperately hangs onto to his belief by reproducing a strawman conspiracy. Nevertheless this is a big first step for George.

    http://www.monbiot.com/archives/2009/11/23/the-knights-carbonic/

  46. Steve at the Pub

    Katz #40 You just put yourself on the wrong side of history. Bad judgement call. There are already changes. Not only persons saying they are no longer blind believers, but a significant reduction (possibly to zero) of the phrase “the science is settled”.

    Religious zealots (ie blind believers) will continue to put their hands over their ears & chant “can’t hear you”. Believers who wish to retain credibility are saying stuff along the lines of “what those emails really meant to say was…”

    Those who haven’t invested all their intellectual reputation in the climate change religion are more circumspect.

    Summary of “Climategate”, (perhaps it should be known as “Manipulation-gate”):
    Indisputably, “the Debate is over” has changed to “the Debate sure ain’t anywhere near over”.

  47. tssk

    Given all this debate surely it’s nuts for Turnbull to go ahead. He needs to put the brakes on this now.

  48. Paul Norton

    Our prejudices about commenters on MSM blogs are being confirmed at this very minute.

  49. Katz

    Katz #40 You just put yourself on the wrong side of history. Bad judgement call. There are already changes. Not only persons saying they are no longer blind believers, but a significant reduction (possibly to zero) of the phrase “the science is settled”.

    Name them.

  50. Steve at the Pub

    Paul Norton, I have a news bulletin for you. Most people, not just “commenters on MSM blogs” are opposed to new or increased taxes. Always.

    One needs no knowledge of taxes, blogs, commenters, or MSM for this. Just a passing experience with human nature.

  51. Katz

    I guess, in the light of PN’s news, the Shadow Cabinet has also decided to drift over to the wrong side of SATP’s history.

    Gee, it’s getting more crowded all the time over here on the Wrong Side.

  52. Sam

    Paul 48, All this MSM commentators harrumphing about how they’ll never vote Liberal again is just hot air (carbon-free, of course). In our electoral system, they have nowhere else to go, ultimately. What are they going to do, preference the Labor Party?

    Their only hope is that the Liberal Party goes to war with itself, and the sceptics win. But that will be a long and a bloody fight. Years and years of opposition await, the Liberal Party might be split interminably …

  53. tssk

    There is another option.

    The data is found to be a fraud or worse a conspiracy.

    The Governer General seeing the path Australia has been set on will lead to it’s destruction calls in Rudd and dismisses him.

    Turnbull is installed as the next Prime Minister. In the face of the new evidence recants his position and leads Australia into a golden age of prosperity.

    The ALP discredited and in ruins slowly disappears much like the Dems.

  54. David Irving (no relation)

    amortiser, I reckon Mobiot was just pulling your leg.

  55. Katz

    The Governer General seeing the path Australia has been set on will lead to it’s destruction calls in Rudd and dismisses him.

    Failing the GG, perhaps the Tooth Fairy.

  56. Nickws

    Sam @ 52:
    All this MSM commentators harrumphing about how they’ll never vote Liberal again is just hot air (carbon-free, of course). In our electoral system, they have nowhere else to go, ultimately. What are they going to do, preference the Labor Party?

    Hmmm, didn’t the Hansonites in ’98 do a lot of preferencing of Labor candidates as a way of telling `Little Johny Coward’ just how they felt about the GST, the guns laws, etc.?

    Of course that could never happen now, what with Malcolm Turnbull having the fervent loyalty of the Steve At The Pubs of this world {not, duh}

  57. Ootz

    DI (nr) It’s an outrage, a scientist has got humor! Some body notify Mr Bolt, quick.

    Jeez DI like you, I hope no one is going to dig up my remarks buried in COBOL all those years ago (blush).

  58. David Irving (no relation)

    Yeah, Ootz, we’ve all got some dirty little secrets buried in source code files.

  59. Howard Cunningham

    I have a question.

    I haven’t read (any maybe I skimmed over it) an explanation as to who “adjusting data” is.

    I would expect most people would think scientists conduct experiments, or testing, or study something, and a result comes out. Adjusting the data “sounds” like you didn’t like the result, so you therefore change the result. An explanation would be really handy. I’m sure it’s necessary, and probably not completely compatible with my overly simplified precis. But then again, in Year 10 Science I argued that rust was caused by the acidity of salt.

    So my question is: what is “adjusting data” in this context, and exactly why (and in what circumstances) do scientists do it?

  60. Ootz

    Yeah DI, shock horror ….. medications come off shelfs, bridges are going to be closed, banks are offering apologies, air planes grounded ……… programmers have found to be leaving incriminating comments on programs. GOSH, who needs global warming for a scare!

  61. mitchell porter

    I am shocked by this detail. Someone had hacked the RealClimate server, blocked access to other users, uploaded the zipfile, and cued up a post announcing it to the world. Moreover, this was done from a Turkish IP address, while announcements in comments at skeptic blogs were done from Russian and Saudi IP addresses.

    I don’t know what software RC runs on, but that’s a fairly sophisticated degree of penetration. Most website vandals just blank the homepage and insert their own text. Moreover, RC is the center of the orthodox climate blogosphere. It’s one thing to go fishing for known security holes on websites, and to take over the ones where you get lucky. Being able to compromise the central site of the climate blogosphere suggests an ability to choose the site and then find the way in, which would imply a much higher order of hacking ability.

    I suppose it is possible that someone just went fishing for vulnerabilities at important research centres like CRU, got lucky and found RC passwords. (I’m not sure whether anyone from CRU posts at RC; if so, the economical explanation is that CRU was the weak link, and access to RC was a lucky side-effect.)

    As for the IP addresses from three different countries, it sounds like either someone is faking their apparent Internet location, or they are using a network of compromised computers.

  62. tssk

    Surely though if the information is correct the hackers are heros in stopping us all from wasted time and effort. (Oh and Katz…I know some old timers who still can’t believe Whitlam was tipped out. I’m betting the Libs are in a tight enough spot to call in a few favours although the irony of Turnbull being put into power by a representative of the Queen would be massive.)

  63. Roger Jones

    For those who want a little more background to help in interpreting what’s written in the emails (HC, there’s some info about adjustments in there) – by the way, they’re not written with me intending to persuade people one way or the other.

    The language about “tricks”, “fixes” and “hiding” data (in full view, mind you) is typical of a group of climate nerds and as such is playful and ironic

    The concern over FOI for climate data and also MAGICC is because both are only used under licence. CRU licenced MAGICC until it transferred to NCAR with Wigley (around 2003-4, I think). The licence was to protect research (non-commercial) use and to ensure appropriate acknowledgement. The executable was freely given to researchers who signed the licence. Wigley does not and will not make the code freely available and code is made available to only a few researchers on request.

    Climate data is commonly held on licence with the proviso that it not be passed on without permission. CRU’s global data sets are constructed on this basis. Again, the licence is non-commercial.

    WMO best-practice methods are used to adjust climate data of observational artefacts. CRU is involved in developing new methods that later become common practice (though they are not the sole group who does this)

    As such, the data-sets adjusted and compiled by CRU have been verified independently – especially by the GISS data in the USA. I have done such work for Australia, which was published in Folland et al., in GRL in 2001.

    Because the adjustment of data sets has been going on for years, the process is very complex and cannot easily be reproduced. There are many different data types and they are adjusted by varying amounts at various times. Because those in the business know this, they know that demands for the source code and all the adjustments, given data permissions etc just cannot be complied with in any practical sense without devoting person-years to fulfilling those requests.

    Separate adjustments are made for station moves, instrument changes, shelter changes, bucket types for sea surface temps, locational effects for ships of opportunity, urbanisation and so on. This for temperature. For rainfall, there are several dozen different raingauges used world-wide ad they all catch different amounts of rain.

    Opposition in the emails to making data and code freely available (which is a development of the past five years and is largely a reaction to research on pharmecuticals) is based on the view that many requests/demands are not for bona fide research. It is also because there is a great deal of legacy in these methods as noted above.

    Behind the barricades attitudes have come principally from attacks on Santer (Post SAR), Mann and Jones. These attacks have now been ongoing for two decades and come from both funded and unfunded quarters. The attackers, because they have private funding, never have to declare their sources.

    These attacks and vexatious requests take researchers away from their science, which is about the worst thing that can happen to a researcher (in their opinion) and accounts for the grumpiness of the emails. Whereas requests made in good faith have always been handled as quickly as possible. This is a highly collaborative area where data is frequently exchanged between individuals and groups (subject to research licence).

    CRU is still seeking all the permissions to make third party data available on the more recent basis that methods and processing be made freely available, but it is difficult to arrange all the necessary permissions. However, the results have been available for some years and researchers are free to compile their own data sets from national climate data archives.

    The Climate Research episode (used as a sign that scientists want to interfere with the refereed literature), was itself a mis-use of the refereed literature and prompted the resignation of most editors of Climate Research, a subsequent loss of reputation and reform of the editorial process of that journal by the publishers. This has been well documented but is not common knowledge.

  64. Baraholka

    The CRU hacking is not an own goal.

    It wil successfully muddy the public perception of AGW and buttress the denalist position for some time to come, as it has created a new source of Hannah’s Balls, the bounciest being ‘Hide the Decline’ .

    It’s a soundbite par excellence.

  65. David Irving (no relation)

    From what you (and others) have said, Roger, it sounds like the data manipluation causing so much faux horror is nothing more than what is called “normalisation” in some circles – in other words a completely legitimate way of dealing with a large heterogeneous data set.

  66. Blair

    Developing techniques for temperature data adjustment is a large part of my scientific existence, so I’ll give a simple example of how it is done.

    At Port Macquarie, the observing site moved from a location in town, a kilometre or so south of the town centre and a few hundred metres from the ocean, to the airport which is several kilometres inland. The two sites operated in parallel from 1995 (when the airport site opened) to 2003 (when the town site closed).

    During that period of parallel observations, on average maximum temperatures at the airport were 0.6 degrees warmer than those in town, and minimum temperatures 1.5 degrees cooler. It follows that, if you wanted to make the pre-1995 town observations consistent with the post-1995 airport one, at the most basic level you could adjust the town maxima up by 0.6 degrees, and the minima down by 1.5 degrees.

    If you wanted to go to a more sophisticated level you would calculate adjustments separately for each month; as it happens the maximum temperature difference between the two sites is largest in summer and near zero in winter (because the sea breezes that cool the coast on summer days aren’t much of a factor in winter), so to take the seasonal variation into account you would need to make a larger adjustment in summer than winter.

    At a more advanced level again, in some places the effect a site move has is greater (or less) on hot days than it is on cool days (on a hot day, the coastal strip can be 10 or more degrees cooler than a few kilometres inland, whereas on a cold, wet day the difference is often minimal), so different adjustments may be needed for hot days and cold days. I’m currently working on developing a data set which implements this technique across the Australian network. (We aren’t always as fortunate to have the amount of overlapping data that we do in the Port Macquarie case, so some of the adjustments have a fairly high level of uncertainty).

  67. Paul Norton

    The pictuer that is emerging is increasingly one of a beat-up seeking to take advantage of the fact that the general public, even when well informed at a lay level about the broad picture of science, don’t know the fine detail of what scientists do and the way in which they communicate with each other when they’re doing it.

    Blair #66, you stand to cop it coming and going. If you weren’t adjusting the figures you’d be pilloried by the denialists for ignoring localised variations between the town and the airport, but since you are adjusting the figures to account for these difference you risk being pilloried by the same crowd for fudging the figures.

  68. Mole

    Blair

    Good explanaition of why its done.

    My concern is with the admission that data is (due to legacy?) still being inputted when the person doing the work has no idea where they came from, what has been done to it in the past, but a vauge idea that the data should show warming in the long term. I would believe the man years of time needed to clean it up and make it transparent are absolutely vital if you want to re-order global economies.

    Building projections on old code/data, without knowing what old adjustments have allready been made is liable to be badly let down by a reviewers inherent biases.

  69. tssk

    And this is why we are doomed to fail. Witness Blair’s post at 66. Well written. Explains variances in data quite well.

    But screw that, TL:DR Hide The Decline for the win.

    As I’ve said before, the only way to deal with this now is to tighten your own belts energy wise. At the very least you’ll have more money to buy beer.

  70. Mole

    tssk

    Actualy for the whole scheme to work you will be paying more for your beer, or spending the same amount and drinking less.

    Its not a “have your cake and eat it” thing at all. For the ETS (and other schemes) to work it has to force people to pay more for less.

  71. zoot

    My concern is with the admission that data is (due to legacy?) still being inputted when the person doing the work has no idea where they came from, what has been done to it in the past, but a vauge idea that the data should show warming in the long term.

    Could you be more precise please? If you are referring to “Hide the decline” you are completely misinterpreting the evidence.

  72. Mole

    What I mean is the bloke doing the programming is trying to make sense of data with no “tags” attached to it. Even with the best will in the world I fail to see how he could extract accurate data.

    So he goes back and tries to hammer it into the same shape it was before so no “mistake” shows up.

    He may be the best at his job and doing it as “straight” as possible, but we are being asked to regaurd tenths of a degree as significant in a model which (if you read through the notes), may end up being out by .5 of a degree or more.

    Hes coming close to fitting the data to the outcome, almost certainly not deliberately, but his fustrations at “incorrect” data skewing his results indicates he may be falling into his own bias.

  73. zoot

    Thank you for replying Mole, but which bloke are you referring to? He must have a name, or at least a position you can identify. What is the data with no “tags” attached to it? (temperature?, rainfall?, measured CO2?). What constitutes a “tag”? I really have no idea what you are referring to.

  74. David Irving (no relation)

    zoot, mole may be assuming that all this raw climate data is in a nice little XML file.

  75. kerry west

    I don’t know about America but in England destroying information as a result of a Freedom of Information request will at least end up in termination of employment, and probably prison.

  76. Peter

    I think this skeptic has a fairly reasonable response to the leaks.

  77. Wozza

    Mercurius @20 and others.

    Your argument is a purely semantic one. Of course there are ways of presenting data (eg graphs, or application of statistical tools) to smooth trends, or needs as Blair has pointed out to find ways to compare different data sets.

    This however has got nothing to do with the statement “genuine scientists adjust data to reach their goal” with which you are on the record as agreeing. Data simply cannot be “adjusted” in science, period, let alone in order as you appear to believe in the interests of achieving a pre-determined goal. Any data point – as a distinct, numerical, observation – remains unchanged (can be discarded if there are compelling reasons of experimental method failure, or interpreted as part of a series, but not fiddled).

    There are too many people round here who are prepared to make sweeping statements about “the science” being settled when they have clearly no experience of what hard science actually entails (as opposed I suspect to the social sciences where they are more comfortable)

    It is interesting that Michael Mann has felt it necessary to say in relation to the e-mail affair that he “did not manipulate any data in any conceivable way.” Which is it to be – the data was not manipulated, as a climate scientist of repute (at least around here) thinks is the correct answer, or it was, but hey, that’s OK because manipulation is an integral part of the scientific method, happens all the time?

    For the record I reserve judgment on the authenticity of the e-mail material. It is interesting though that none of the e-mail authors, at least to my knowledge, has yet denied they are genuine.

  78. Wozza

    Brian @42

    This story goes to the heart of the problem, for those of us who believe in greenhouse warming but are doubtful about its allegedly apocalyptic end point. I don’t think Carter was necessarily dissembling , except perhaps in a small sin of omission.

    Your key sentence is “It’s just that someone had inserted 0.4C as the value for climate sensitivity, the amount of warming for the doubling of CO2, instead of the more usual 3C for short term feedbacks as per IPCC AR4.”

    As I understand it, 0.4C is accurate as the temperature effect of direct greenhouse forcing. 3C as you say depends on assumptions about feedbacks, and depends on …… modeling.

    The critical issue has always been the models, their accuracy, and the assumptions on which they depend, and there is plenty of room in my view to believe that there is considerable doubt on this score, which I won’t go into here as it is not the thread topic (though in relation to the thread it is interesting that one of the e-mails has Trenberth saying almost the same thing – the departure of recent temperatures from the models is a “travesty”).

    Perhaps Carter should have made clear where 0.4C comes from, but equally when 3C is used the basis of this figure should be made clear (as you have, but it often doesn’t happen).

  79. Pterosaur

    Perhaps this may cure the idiot frenzy of the denialati ?(From East Anglia)

    Not that I would expect them to actually consider facts…..but one may hope.

  80. zoot

    Wozza, all of your points have been answered (see, for example, Blair above). On the previous thread I quoted Gavin Schmidt’s reply to those who believe climate scientists study global warming. I can’t be arsed digging it out again, but to paraphrase, they don’t. They have their field of study and whether the planet is warming or cooling that’s the field they study.

  81. zoot

    University of East Anglia’s response is online.

  82. Mole

    Here a summary of commentary left on the code that gave us this.

    http://www.anenglishmanscastle.com/HARRY_READ_ME.txt

    Thats from this bloke, Ian Harris (youll find his name in the info bar on top of the page) who is Mr. Ian (Harry) Harris, Dendroclimatology, climate scenario development, data manipulation and visualisation, programming, at the CRU.

    He is the poor sod trying to make decades of mismatched data “work”.

    Some of his notes include.

    “..So what’s going on? I don’t see how the ‘final’ precip file can have been
    produced from the ‘final’ precipitation database, even though the dates
    imply that. The obvious conclusion is that the precip file must have been
    produced before 23 Dec 2003, and then redated (to match others?) in Jan 04…”

    “..In other words, the *anom.pro scripts are much more recent than the *tdm
    scripts. There is no way of knowing which Tim used to produce the current
    public files. The scripts differ internally but – you guessed it! – the
    descriptions at the start are identical. WHAT IS GOING ON? Given that the
    ‘README_GRIDDING.txt’ file is dated ‘Mar 30 2004′ we will have to assume
    that the originally-stated scripts must be used..”

    “..So, to me this identifies it as the program we cannot use any more because
    the coefficients were lost. As it says in the gridding read_me:

    Bear in mind that there is no working synthetic method for cloud, because Mark New
    lost the coefficients file and never found it again (despite searching on tape
    archives at UEA) and never recreated it. This hasn’t mattered too much, because
    the synthetic cloud grids had not been discarded for 1901-95, and after 1995
    sunshine data is used instead of cloud data anyway…”

    “..Decided to go with the ‘spc.94-00.0312221624.dtb’ database, as it
    hopefully has some of the 94-00 normals in. I just wish I knew more.

    Conversion was hampered by the discovery that some stations have a mix
    of % and % x10 values! So more mods to Hsp2cldp_m.for. Then conversion,
    producing cldfromspc.94000312221624.dtb. Copied the .dts file across
    as is, not sure what it does unfortunately (or can’t remember!)…”

    “..These are very promising. The vast majority in both cases are within 0.5
    degrees of the published data. However, there are still plenty of values
    more than a degree out..”

    Thats a small, tiny sample of whats in his log.

    Thats the “who” I was refferring to, and yes after 20 years I would like the data sorted out into one format.

    So to sum up, the bloke at the CRU has been making data “fit” because his original sources are lost, corrupted, mislabeled or missing. Whole variables (such as cloud) drop in and out and are “synthisised” to match.

    Thats why the emails are such a “big issue”, its the data files that show bits like this rather than the correspondence that is really shocking. 90% certainty my hary bum…

  83. Wozza

    Mole, thank you, that is indeed interesting.

    Pterosaur – perhaps you would like to look at Mole’s comment carefully, and consider whether what it quotes is consistent with the assertion that “There is nothing in the stolen material which indicates that peer-reviewed publications by CRU, and others, on the nature of global warming and related climate change are not of the highest-quality of scientific investigation and interpretation.”

    That quote is directly from the University of East Anglia’s response which you apparently believe is so convincing that it should lay all argument to rest.

    “Clutching at straws” is far too feeble a phrase to describe a good deal of what has been going on in this blog over the past couple of days.

  84. Zarquon

    Wozza:

    So to sum up, the bloke at the CRU has been making data “fit” because his original sources are lost, corrupted, mislabeled or missing. Whole variables (such as cloud) drop in and out and are “synthisised” to match.

    Trevor Davies, University of East Anglia

    There is nothing in the stolen material which indicates that peer-reviewed publications by CRU, and others, on the nature of global warming and related climate change are not of the highest-quality of scientific investigation and interpretation. CRU’s peer-reviewed publications are consistent with, and have contributed to, the overwhelming scientific consensus that the climate is being strongly influenced by human activity. The interactions of the atmosphere, oceans, land, and ice mean that the strongly-increasing concentrations of greenhouse gases in the atmosphere do not produce a uniform year-on-year increase in global temperature. On time-scales of 5-10 years, however, there is a broad scientific consensus that the Earth will continue to warm, with attendant changes in the climate, for the foreseeable future.

    The bottom line is, if you’re not looking at the published data, your just arguing from ignorance.

  85. Zarquon

    Oops, sorry that was Mole I quoted.

  86. Zarquon

    By the way, do any of you know what the data Harris is using actually is? What it’s used for? How it’s analysed? How big the error bars are? How they change after the analysis? How big are the gaps in relation to the total dataset? How the data compares to other datasets?

    Those are just the beginning questions, at least.

  87. zoot

    He is the poor sod trying to make decades of mismatched data “work”.

    Total misrepresentation.

  88. Wozza

    Thnk you Zoot for that penetrating analysis.

    I now see that I have been wrong all along. Forgive me.

  89. Mole

    zoot.

    Read the linked file, it runs to over 1000 lines, just pick a couple of random points and tell me it seems like a rational basis to restructure economies on.

    Its not from some crackpot, but the man whos attempting to make the data work with the tools he has.

    It doesnt invalidate AGW as a thory, so theres no need to “die in a ditch” defending this piece of work. However it does show substantial sums of money need to be spent reviewing and rewriting vast amounts of info which has been put out as “acurate withing 1/10th of a degree.

  90. zoot

    Read the linked file, it runs to over 1000 lines, just pick a couple of random points and tell me it seems like a rational basis to restructure economies on.

    No economy is going to be restructured on the basis of Harry’s code comments. Get a grip.

  91. Zarquon

    has been put out as “acurate withing 1/10th of a degree.

    Total misrepresentation.

  92. Mole

    Keep clutching at those straws…
    zoot
    These analisis’ (and a number of others), make up the backbone of the UN climate panels reports. They arent scribblings at the margins, they are the guts of the deal. That UN council is making policy based on data put through this process, does the process seem best practice to you?

    Zarquon
    Just how is that a misrepresentation, seriously how?

  93. Mole

    As a quick link, heres wiki. You can knock yourself out looking at the peojections all prettied up to 1/10th of a degree to make it appear more scientific.

    http://en.wikipedia.org/wiki/IPCC_Fourth_Assessment_Report

  94. Zarquon

    The raw weather data don’t need to be accurate to within 0.1C to find a statistical trend in climate. The problems with the base dataset will contribute to the error margin but much of it will be smoothed out by temporal and spatial averaging used to get a climate trend. Have a look at Wikipedia

  95. Mole

    Zarquon

    Read some of the programmers notes and estimate for me the sort of accuracy youd expect from that botched process.

    He doesnt(at times) know where info comes from, how it relates to each other, etc, until he get it to “fit” with various subroutines and tweaks.

    Half of his problem are trying to smooth and average, he keeps running into glitches, artifacts and muddled results. The information generated might be ok as an inaccurate general trend, but to assign confidence levels of 90% to that is just foolish.

  96. zoot

    Mole, do you have any experience in reading programmer’s comments? Do you have a coding background, or are you, as you appear to be, a neophyte? There have been many comments from people experienced in this area on this and other blogs such as Deltoid. You’ll sleep easier at night if you seek them out.
    Meanwhile, the planet continues to display the effects of warming in line with IPCC projections.

  97. Mole

    Heres the funny thing, I can read deltiods experts or I can read proclaimed experts from other sites.

    Neither agree on the significance of the notes. As per usual the divide is allong the pro/anti AGW lines.

    However only one side is asking for the introduction of a new taxation regime and the effective lowering of “western” lifestyles. Id say the onus is on them to be squeaky clean not the other way around.

    The claim being made is huge and complex, the solutions proposed world changing, certainty about the data is a minimum requirement. I cant see any signs of that clarity in the dogs breakfast in those notes.

  98. zoot

    I cant see any signs of that clarity in the dogs breakfast in those notes.

    Which is why nobody is depending only on those notes. If all the CRU research was removed from the corpus there would still be an overwhelming scientific case for limiting future warming of the planet. How that limitation is achieved is a political issue which will be worked out by the politicians at Copenhagen (and wherever else they meet in the future). Don’t confuse the political with the scientific.

  99. Mole

    Zoot.

    Then why are you and a number of other commentors above attmpting to explain it away.

    I mentioned above this data set by itself doesnt invalidate the AGW theory. Not by a long shot. But if one mob have (for whatever reason) “sexed up” the figures then they need to look at some of the damage that has done to other researchers who may have been using some of the same data in good faith.

    This one group should be quarantined until a review is done of their work. And unfortunately anyone who has used their information/figures has the heartbreaking task of seeing if their mistakes have contaminated their research.

  100. Pterosaur

    1.There are a number of research institutions around the world conducting independent research on the factors which affect climate.

    2. Each of these independent organisations has arrived at essentially the same conclusions as to what the major influences are on our climate, and that we are undergoing a period of global warming.

    3. Each of these organisations has independently concluded that the major cause of the global warming is anthropogenic CO2 emissions.

    4.On this basis it is very hard to see that Mole’s comment that “This one group should be quarantined until a review is done of their work.” is in any way justified, and in all probability represents nothing more than the distasteful practice generally known as “concern trolling”.

    Which is, of course about all the denialist trolls have, given that they cannot produce either data or science to refute AGW, and despite significant efforts can do no more than “muddy the waters” of discussion about remedial action with their faux concerns.

  101. zoot

    Thank you Pterosaur.

  102. Rob

    What’s happening is that we are witnessing the beginning of the death throes of the biggest scam in human history. We are being asked (pace Mole) to destroy a 21st centry economy on the basis of a re-heated version of medieval superstition. The future of the planet does, indeed, hang in the balance, but not in the way the warmists would have us believe.

  103. Martin B

    It would be interesting to see if the general circulation climate models, observation archives and peer-reviewed literature that support the skeptical case could stand up to the same level of scrutiny.

    Oh that’s right, there aren’t any…

  104. Zarquon

    We are being asked (pace Mole) to destroy a 21st centry economy on the basis of a re-heated version of medieval superstition

    I suppose that’s a more novel scare than reds under the bed.

  105. Lefty E

    In that case Rob, Ive got some some excellent coastal and low-lying atoll real estate to sell you. Going fast!

  106. sg

    I don’t think that the comments in these programmes necessarily mean what you think they do, wozza et al. For example, the phrase “decline” refers to a decline in maximum tree ring density, e.g.

    The tree-ring density’
    printf,1,’records tend to show a decline after 1960 relative to the summer’
    printf,1,’temperature in many high-latitude locations.

    (I took that from that horrible Watts up with that blog). If you do a quick search on tree ring density you’ll find that it is a confounding variable in analyses of temperature based on tree-rings, and there are corrections built into the data for situations where, for example, drought causes tree ring density to change more than should be expected given the temperature. Those 1000s of lines of code appear to be Harris’s attempt to reconstruct work that was done on a beowulf 1 cluster before 1996, for use on the new computers. For example (from that Harry read me):

    ..so the data value is unbfeasibly large, but why does the
    sum-of-squares parameter OpTotSq go negative?!!

    Probable answer: the high value is pushing beyond the single-
    precision default for Fortran reals?

    (the “sum of squares parameter going negative” does not mean negative temperatures, but a failed program”). Or this:

    which is good news! Not brilliant because the data should be
    identical.. but good because the correlations are so high! This
    could be a result of my mis-setting of the parameters on Tim’s
    programs (although I have followed his recommendations wherever
    possible), or it could be a result of Tim using the Beowulf 1
    cluster for the f90 work. Beowulf 1 is now integrated in to the
    latest Beowulf cluster so it may not be practical to test that
    theory.

    I don’t know the provenance of these logs, but I think they might be an attempt to reproduce long-finished work for the IPCC review. The work appears to be an adjustment to the tree-ring series to account for tree-ring density values that don’t fit the long-term trend. “Harry” appears to have been reconstructing work Tim Mitchell did before 2004; Tim Mitchell’s works appear to have been published in, for example, 1999.

    Further down we discover that the program is being written to handle “rolled over” stations, such as Blair discussed at comment 66. There is a specific and amusing example of Byron Bay.

    It’s also clear that “Harry” is under a serious time constraint. He is reproducing someone else’s long-ignored coding for a specific deadline post-2004, and checking it against refreshed datasets from 2004-2006, i.e. most of the code which is evidence of “bad programming” is actually one man’s extremely detailed attempts at verifying old work before reusing it. There is no evidence from his reproduction that he has run into anything except badly commented, idiosyncratic code, and all the sections end with him getting to within 5% of the original data, and identifying the cause of the discrepancy.

    This is your gotcha?

    Oh and wozza, if you think data should never be adjusted I suggest you look up “Principal Components Analysis”.

  107. Wozza

    sg, for heavens sake what do I have to do to get my point through?

    Principal components analysis, as you obviously know, is a statistical technique used for finding patterns in data. That is, it is about interpreting the directly observed data. That is a far cry from changing the observed data itself. Some people round here do not seem to recognise that difference. That’s all I’m saying.

    I am not decrying the usefulness of statistical techniques. My partner has a PhD in psychology and does research in that area. She uses statistical programs all the time. I do think however – and so does she – that there is a difference in hardness as it were of conclusions drawn from statistically manipulated data, and conclusions based more directly on observed data. It disappoints me that climate science is conducted so often with techniques typical of the social sciences, not the hard sciences.

    Given the complexity of climate, I am no doubt over-optimistic to expect anything else. I am merely trying to get through to some people that climate “science” is far from being a hard science like, say, physics, and that that has implications when one taliks of “settled science”.

  108. sg

    wozza, climate science is physics. Principal Component Analysis (PCA) is commonly used for data reduction and the resulting linear combination used in place of the original data. This linear combination results in a smoothing of the originally observed data, and a loss of information. It’s a commonly used, very old, and respected form of adjustment. I don’t think that the CRU kids use it, but it’s one example of a suite of techniques.

    Someone at deltoid said it better: if I have 10 thermometers and one of them reads 1C higher than the other 9 during calibration, I subtract 1C from that thermometer. What do you think happens when you zero a scale?

    I’ve seen a couple of websites now talking about the contents of these emails as if they were evidence of lying and manipulation. Gavin Schmidt at RealClimate has an explanation for everything to date, usually putting them into the context of published work. That so-called “decline” for example has been in the published literature since the 90s. Schmidt confirms my interpretation of these Harry Harris logs, but I managed to work this out for myself – because I programme in stats packages and I know how to read these kinds of comments. Somewhere in my work I have several lines of comments which simply read “F**king R! F**king R!” (minus the asterisks). I certainly don’t want that sort of stuff being leaked by a malicious hacker!

    All we have here is a bunch of stolen emails, and character assassination. It’s oily and nasty, and it is certainly going to have ramifications for university policy on openness and transparency. Another corruption of the scientific process brought about by the denialists.

  109. Wozza

    sg, OK let’s agree its semantics, around the meaning of “adjust”. I understand your point. I don’t think a number of others understand the difference between what you describe and actually changing raw data, though.

    Climate science should be physics. That is why I am disappointed that it is so often not. The entire paleo-climate argument is not physics in any way, and that field is where the greatest rorting happens.

    Some stunning logic at the end though – the corruption of science at East Anglia is the responsibility of the denialists? The whole point of the current debate is to establish what the e-mails show – or don’t show – about corruption of science by scientists at East Anglia.

    I agree that stealing the e-mails was an undesirable, even “nasty” way of going about things, but lets face it, they brought it on themselves by refusing to divulge information when repeatedly requested and legally obliged to provide it(“I think I’ll delete the file rather than send to anyone”). If you think that East Anglia pre this event was a shining example of university openness, you have a funny definition of open.

  110. sg

    Oh Wozza, the data is available online. The FOI request for code was denied not by East Anglia university, but by the organisation that makes judgements about FOI requests. You are aware of that, aren’t you? But it’s their fault for “hiding”? And what good has come of all this data being available online? How come the discrepancies you think these emails highlight have never been identified by denialists poring over the data? How come the body of denialist literature is so scant?

    Good to see, btw, that you are so big as to admit that stealing someone else’s emails, selecting an incriminating subset, and posting them online, is “nasty” in airquotes.

    It’s also not “semantics” to talk about the meaning of “adjust”. When I measure your weight with clothes on, I adjust it for the estimated weight of clothes. Similarly when I measure the weight of a truck – that’s why trucks have their unladen weight written on the side. If I have a set of observations I know are affected by a physical problem – what Gavin Schmidt calls “non-climate effects” – of a known provenance, I adjust for it. Scientists do this all the time, including physicists. Do a search – you’ll find physics is riddled with it. Ever heard of “renormalisation”?

    Also, I think if you look through the world’s physics departments you’ll find atmospheric physics right there. They’re the people who measure temperature and “adjust” the data.

    This is a sad little episode and the people defending it need to take a long look at themselves.

  111. murph the surf.

    SG wrote “This is a sad little episode and the people defending it need to take a long look at themselves…”
    .
    Indeed!
    http://www.guardian.co.uk/environment/georgemonbiot/2009/nov/25/monbiot-climate-leak-crisis-response
    .
    I don’t think Monbiot is arguing from a poor perspective – it is rather a challenging one however..

  112. sg

    I’m actually really disappointed in Monbiot’s response to this. He has fallen hook line and sinker for malicious slander. The examples he gives are pathetic attempts at cooking up a storm and he shouldn’t be responding like this. He should be instead talking about what a crock of lies the outrage is. As for falling for this notion that the scientists – who subject their work to peer review and have made their data publicly available – are engaging in secrecy and suppression – this is pathetic.

    Monbiot needs a cold shower, or a becks and a good lie down.

  113. Wozza

    sg you are reducing the “adjustment” issue to absurdity. When you weigh a truck empty, and then again full – or a person with and without clothes – you have TWO SEPARATE OBSERVATIONS AKA DATA POINTS. If you then subtract one from the other, that has nothing remotely to do with “adjusting” either piece of data. There no point in pursuing this argument further if you are reduced to that sort of stuff.

    As for the argument that the University of East Anglia has no responsibility for denying FOI requests for data – that some vague, Orwellian “organisation that makes judgements about FOI requests” frustrated the brave attempts of the academics to uphold their outstanding standards of openness – you are aware that my earlier quote ((”I think I’ll delete the file rather than send to anyone”)is from Phil Jones? And of who he is?

    And in any case there are far more worrying issues than mere denial of data raised by this affair which you are avoiding.

    However I am not going to bother further. I will retire to beat my head against a brick wall. I will clearly make far more progress in that task; walls are, compared to some things, rational

  114. Brian

    Wooza, your approach is simplistic in the extreme and beating your head against a brick wall is not likely to produce more nuanced thinking.

    Go back to Roger Jones @ 69 and read it carefully.

  115. Jay

    Well, thank goodness we have that emmenent climate expert in Rob @ 102 here. Thanks for setting us straight. I await your peer reviewed research appearing in Nature shortly.

  116. David Irving (no relation)

    sg @ 112, I reckon you’re right – a Becks (or 6) would be much nicer than a Bex. (As an aside, I note that APCs are no longer the recreational drug-of-choice of middle-aged housewives.)

  117. Brendon

    sg @112“I’m actually really disappointed in Monbiot’s response to this.”

    Monbiot is a real facts and figures writer. He loves sticking it up his opponents using their sloppy inaccurate and contrived contributions. He probably feels stabbed in the back by his own allies and people he felt he had supported.

    Would have been an idea if he waited a day or two. Monbiot might have to apologize twice in the same week.

  118. sg

    Wozza, storming off in a huff doesn’t change the fact that data adjustment is normal in this field. You’ve been given multiple examples of routine scientific practice. If you don’t like them it doesn’t reflect on the procedures the people in these emails used, only on your misunderstanding of how science works.

    When I look at the response to these emails by the skeptics I see a lot of that going on. Watching people assiduously insisting that the inner workings of a scientist working on reconciling old data with new data is sure proof of fraud is very unpleasant. there but for the grace of god go I…

  119. Martin B

    “I’m actually really disappointed in Monbiot’s response to this.”

    Also you can hardly expect a journalist not to get quite touchy about FoI issues.

  120. Brian

    I left the public service just as FOI was being introduced. I understand democratic rights and all, but I saw its effects on the efficient working of honest public servants as almost wholly negative.

    Whenever you write something you have a particular audience and purpose in mind and adjust your style accordingly. If you have to do this in a goldfish bowl you have to regard the people potentially gawking into the bowl as part of the audience. When you know that they bear you ill will and can’t possibly understand the circumstances within which you operate, your purposes and procedures, well, I’m glad I left when I did.

  121. Wozza

    sg, for the record I have a PhD in a hard science (physical organic chemistry). I am well aware of how real science works, and that it is not how at least some of climate science works. I have made a number of specific points– the Phil Jones admission, the demonstrable fudges in much of the paleo-climate work, your ludicrous truck analogy, etc – about what you have alleged , and you have simply ignored them in favour of generalised patronisation and assumptions of superiority

    If I stormed off in a huff, as you put it, it is because it is impossible not eventually to get weary of arguing with someone who is merely talking to himself/herself.

    Brian I agree that Roger Jones makes much better arguments – not hard – than sg, but they are not wholly convincing. “Typical of a group of climate nerds …. playful and ironic” means “er yes, I agree it looks damning”. The licence/data concern is valid – but similar arguments about legal requirements for confidentiality (eg in regard to public/private partnerships) have been regularly condemned around here, so there is a bit of selectivity in giving them a free pass this time. Reluctance to provide material to people who will just beat you around the head with it is understandable – but it is contrary both to the law of the land (FOI) and the usual standards of scientific openness.

    I remain convinced that many commenters in LP do not understand what “science” means, and don’t even care. And if we are to start using terms like “lazy…,grow up” in this discussion, Brian, then I will say that I certainly include you in that number. Mindless regurgitation of talking points from politically approved sources and total ignore of anything from elsewhere does not make for either knowledge or understanding.

    It is revealing that there are currently 235, mostly gloating, comments in the two threads on the ETS and the Liberal leadership, and 34 in the thread on Copenhagen. For most people round here, climate is mostly a battleground in tribal politics, and hardly at all an issue about science.

    Over and out.

  122. Fran Barlow

    Wozza observed:

    For most people round here, climate is mostly a battleground in tribal politics, and hardly at all an issue about science.

    That’s because the basic science is settled. What is most germane is how we respond (the quality, ubiquity and quantity) and how soon. The apparent decomposition of the coalition over this issue threatens to clear the way not only for an effective mitigation scheme (if not to take to Copenhagen then certainly to the next election possibly in March 2010), but for new policies in other areas as well.

  123. Brian

    Wozza, I suggested that you were being lazy because you were. The two readily available sources I referred you to (Skeptical Science and Monash) are not “politically approved” but based on a collection of the scientific evidence. If you read them, or similar, and then post saying you have queries or disagreements based on something more than opinion, I’m interested.

    If I was sharpish, I’m sorry, but it does try the patience when old chestnuts are repeatedly served up when the relevant information is readily available.

  124. Brendon

    Wozza,

    Re the email scandal: Wouldn’t a control group to see if there was anything to the East Anglia email saga be other scientific bodies?

    If the people at East Anglia were fudging the figures then it would show up with publications coming out of places like NASA. But it doesn’t. If NASA is right, then East Anglia can’t be making it up too. Not in any meanigful way.

    http://earthobservatory.nasa.gov/Newsroom/view.php?id=38818

  125. sg

    wozza, they’re not demonstrable fudges. I’m sorry, but they’re just not. talking about them as if they are and using these emails as gotchas on basic scientific process is just obfuscation for ideological reasons. Finding an email by a scientist concerned about an FOI request is hardly proof of fraud, and some random bravado about “I’ll delete the file” is not evidence of a criminal conspiracy.

    You’re clutching at straws and misrepresenting the work of the scientists involved while you do it.

    Also my examples aren’t silly or patronising – you haven’t answered a single calibration example, and why not? You haven’t given any response to the point about non-climate effects on temperature readings, and you haven’t done so because you know that any attempt to do adjustments of the sort Brian described above will inevitably involve modelling. If you have a problem with modelling you’re just out of step with modern science.

  126. Ootz

    Wozza, sorry to see you go in such a state.

    I genuinely believe in rigorous debate to come to improve science as life in general. There is some truth in your statement of “gloating comments”. However, you will find there is also a large percentage of comments in there, which are clearly either trolling or peddling unsubstantiated and uninformed claims cut and pasted from non science websites clouding this debate. What ever the motive or maturity of these posters are aside, but it places ‘skeptics’ such as you into a unenviable position.

    While you maybe disappointed in the level of scientific knowledge and understanding on this blog, where do you suggest is there a higher level of integrity in the debate of science on this highly complex and emotive as well as political issue? Further, I may point out that comments, such as yours re hard science are colloquial and do not contribute well to improve science. They remind me of refectory arguments of old times with members of certain faculties, where I had to defend my ‘science’ by pointing out that as a undergrad I do same if not more subjects in stats, research methodology and ethics and my first written assignment was a lab report of an experiment done in lecture. The rigor of approach and method in science is the badge I wear not level or faculty it was attained in.

    This is where the argument ought to be. First and foremost, validity and reliability of research, adherence to established research methods and statistical procedures as well as individual and institutional integrity. I would argue that in this debate integrity is the central issue. As it has been pointed out, if we cant trust the old scientific establishments and peer review, what can we trust in? There is always the danger of lets say theoretical as well as commercial bias. If we cant trust climate science how about genetic engineering, materials health and safety, medical science et al. Of course we have to be critical as well as sceptical in assessing any scientific endeavor.

    So how do we keep science honest. While FOIs have a place, I would argue that we have not had the debate about the usefulness of such vehicles to improve transparency and integrity of science. (I am more concerned about the security of scientific data and the precedent this ‘breach’ has set.) As for the integrity of ‘ironic and playful’..’climate nerds’ and their antics, well we have to trust the integrity of their supervisors and institute they work for. It certainly does not look good. However, I do remember as a young professional ‘hotshot’ my social skills were not exactly on par with my technical and commercial abilities and skills. Live experience and hindsight are extremely effective in that regard. Integrity is an invaluable bench mark to assess the various characters and scientific information available and I have to say, there is no debate about which camp is more credible.

    The same I would argue for this blog site, aside of some personal indulgences and ignorance to which I am the first to confess. I would agree with you on the gloating aspect, which could include the name of the thread. But Brian has so far made a serious attempt to stay above the fray and encourage a broader debate. As this is a non moderated log, literally everyone can stand on her/his soap box here and some times it shows. It is to Brians and contributors credit that debate has not degraded to the level of MSM blogs and talk back shows. If you really want to see no understanding and knowledge in science, perhaps you should sample some of these sites and try to argue your ‘hard science’ point there.

    I do respect your input to ‘keep the bastards honest’. You may have to get a thicker skin and perhaps rely less on ‘hard’ science and more on the methods and integrity of science in this debate. Me thinks you’ll find a richer quarry all around.

    Cheers Ootz

  127. Brian

    Brendon @ 124, I understand that there are four major groups in the world monitoring global surface temperatures. Each one comes up with something that looks like this one from NASA GISS. So you couldn’t cook the books without a massive conspiracy, of which there is not even a hint.

    On that graph you will also notice red data and black dots. The red dots are from land stations and the black satellite data, another check to show that they are on track.

    Much is often made of the problems of heat islands and the changed conditions of many land stations in the US in particular. This is one of the reasons why the data is normalised. I recently read that a plot of the temperatures using the subset of the “good” stations gave a graph which was virtually identical to the graph from the whole set. The “crook” stations are necessary because they show changes at that site and enable scientists to connect all the dots in producing a graph of temperature anomalies.

    Taking a broader view of the period of the instrumental record, all four graphs look something like this. This graph is much more important than the old “hockey stick”, which uses proxies and is a northern hemisphere (if that) measure.

    Much of the heat, I gather, relates to how Mann and Jones patched the link between the proxy record and the instrumental record to make a graph that visually made sense. I gather the “gotcha” email goes back to the original exercise in 1998.

    Mann and co have been criticised for the statistical methods they used way back then, and in my not totally informed view it seems as though some of this criticism was justified. But Mann and co have since had another go, and so have others. And guess what, they’ve come up with results that look remarkably like a hockey stick. Have a look at Skeptical science:

    Sceptics say that IPCC AR4 dropped the hockey stick. My memory is that they did, but replaced it with a graph that looked more like an ice pick.

    See also the Monash site:

    Probably false, but irrelevant anyway.

    “False”, please note, refers to the statement “It was warmer during medieval times”.

    Which is pretty much what I think about it. And raking over what might have been said back in 1998, however it reflects on the personalities of the actors, doesn’t have much to do with our understanding of the science.

  128. Philip Machanick

    Yes, a few things in the emails look bad. But you have to remember that saying something in an email is not the same as doing it. Also, much of this is communication between colleagues who do not feel the need to explain every detail to each other. Reading between the lines and interpreting is therefore about as useful as reading tea leaves.

    On the other hand the absolute feeding frenzy of the denialosphere is something to behold, and much of it utterly dishonest. An example: one of the stolen files (osborn-tree6/briffa_sep98_d.pro) contains a program for generating a graph with the following juicy lines:

    ;
    ; Apply a VERY ARTIFICAL correction for decline!!
    ;
    yrloc=[1400,findgen(19)*5.+1904]
    valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,- 0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,$
    2.6,2.6,2.6]*0.75 ; fudge factor
    if n_elements(yrloc) ne n_elements(valadj) then message,?Oooops!?
    ;
    yearlyadj=interpol(valadj,yrloc,timey)

    This is conveniently snipped at this point. The rest of the file shows very clearly that THIS CALCULATION IS NEVER USED. Here is the only place where it could have been used:

    ;filter_cru,5.,/nan,tsin=yyy+yearlyadj,tslow=tslow

    Note the “;” at the start of the line, which turns the rest of the line into a comment, i.e., for those who aren’t programmers, it’s ignored when the program runs.

    The fact that I first encountered this very dishonest snippet on the web site of a person very well known in the programming world (http://esr.ibiblio.org/?p=1447) makes it rather hard to believe that the dishonesty was not deliberate.

  129. Brendon

    Brian @ 127,

    I followed your links and ended up in Siberia trying to figure out trees rings. Steve Mcintyre claims Briffa created a hockey stick graph (pointing to GW) out of too small a data set tree-ring sample chronology:

    http://dotsub.com/view/19f9c335-b023-4a40-9453-a98477314bf2

    Then Briffa replies, and it seems Mcintyre got nearly everything the arsed way around.
    http://deepclimate.org/2009/10/30/briffa-teaches-but-will-mcintyre-ever-learn/

    The funniest bit is, where in order to prove the hockey stick graph wrong, Mcintyre gets his own data and comes up with a graph that has temperatures actually dropping in the 20th century. Instead of saying: “Hang on??!! Oops. I think I stuffed up here!”, he points to this as proof(!) the hockey stick graph is all wrong. Apparently I didn’t I miss anything by not going to uni.

  130. Brian

    Philip @ 128, unfortunately that kind of direct dishonesty keeps coming up all the time.

    Brendo @ 129, you did well. To be frank, I just saw McIntyre’s ‘correction’ which looked pretty wild compared with all the serious efforts and left it at that.

  131. Brian

    Counterpoint did themselves proud in interviewing Aynsley Kellow,
    Professor and Head of the School of Government at the University of Tasmania
    and expert reviewer for IPCC AR4 on the great East Anglia leaked emails affair.

    Kellow was more negative and damning than George Monbiot ever would have been. I had the feeling he didn’t have a good grasp of climate science, which didn’t stop him from making some calls only a climate scientist should have made. Kellow, you see, is a social scientist, albeit with a specialty in science policy.