As an Old Data Hand -
I really got interested in this when I saw the data from New Zealand – showing that the raw and adjusted temperatures were showing different trends:

Specifically the raw data was showing no trend and the adjusted data was showing an upward trend. We ODHs know that if you can’t see it in the raw data, it probably isn’t there. Now that’s not always true and sometimes the data needs to be transformed in an amazingly elegant way and suddenly it fits a new pattern. Then you might get a Nobel Prize, or maybe a word of thanks – but it’s mostly it doesn’t happen.

So I was even more interested in ‘Climategate USA’, the revelation that the number of observation stations used in producing the GISS Land-surface air temperature anomalies (Global Temperature data for the newbies) had fallen from over 6000 to about 1500 during the 1990s and onwards. If there’s one thing we ODHs know, it’s that the easiest way to find what you expect in the data is to select the observations that you include in the result.
(The second easiest way is to find something else to normalise it against!)

There are a lot of caveats and speculation after the exciting graph.

So I did a bit of work with the raw data that was made available through realclimate.org (Climate Science from climate scientists) which is the GHCN v.2 (Global Historical Climate Network: weather station records from around the world, temperature and precipitation) . I had previously checked that I got the same pattern from this data regarding the New Zealand divergence and also the Darwin station fuckup, so I am reasonable sure that I am dealing with the right sort of data.
I certainly get the right sort of numbers against the station count.

I thought that if the stations observed vary from year to year, it might be interesting to look each year in terms of ONLY stations that reported in that year and the previous year AS WELL (we sometimes call this a constant sample).

Obviously the absolute temperatures can vary very considerable, if a lot of cold stations were included in Years 1 and 2, but dropped in 3, then the Year 2 absolute would be much lower than the Year 3 absolute. However I can’t really see why the change between the two years shouldn’t be a fairly valid statistic.

I can then apply the change to a base year and compare the trend between GISS, and My Constant Station trend. My data is of course unweighted by geography and has no interpolation. So I include my All Station data (also re-aligned to starting point GIFF base temperature).

I can then take the difference between my result for the constant sample stations and my result for all stations and apply that difference to the GISS temperature data (I’ve used absolute difference rather than an index ) to show what the GISS data might have looked like on a constant sample basis. (GISS+Constant sample)

Dropping the stations?
Notes and queries: Am I comparing Apples and Pears? I don’t really know – I’m using the GISS Global Land Station data so I hope not.

The series starts in 1946 as this is a bit of ground zero in the GISS data I’m using with a zero degree anomaly.

Things I haven’t take account of are probably too many to mention. The number of observations in a year from a station are taken into account in the averaging but if one year is January only and the next July only I haven’t bothered. I reckon these things tend to average out.

One interesting thing is how much more variable year to year the GISS data is than my All Station data – that’s the result of me using the data straight and not weighting the stations in any way (for instance by interpolation). I know this is land temperatures only, but logically I expect global temperatures to be pretty consistant year to year – of course possibly with a trend – because it’s the same whole Earth – is that a wrong assumption?

Of course the same analysis is possible for any country with a reasonable number of sample stations – although I haven’t done it.

For those of you who dont know, there is a big row brewing about the global temperature data (everyone – GISS, CRU and the others) seems to use the same data which I downloaded from http://www.realclimate.org/index.php/data-sources/#Climate_data_raw .

Basically the number of weather stations used was 5000+ up to about 1990 and now its down to about 1500. Do they show a different trend? They certainly appear to.
I divided the stations into those which are live in the data at some point from 2005 onwards and the chart below shows the trend for those stations compared to all the others (from 1910 to 2009). So the ‘All Others’ line goes to 0 from 2005 onwards.

To me this really looks like the smoking gun that Climategate didn’t produce – sorry to all me ‘regular readers’ for using this site to actually produce some data rather than comment on other peoples rubbish.

Dropping the stations?

From the Guardian website:

http://www.guardian.co.uk/society/2009/nov/03/child-obesity-levelling-off

A separate opinion poll yesterday suggested that 50% of obese people earn less than the national average income. The Liberal Democrat shadow health secretary, Norman Lamb, said: “This report makes for particularly disturbing reading as it highlights the worrying link between poverty and obesity. Until we stop trying to dictate policy nationally and give people the freedom to tackle public health problems locally, this cycle of poverty and ill health is likely to continue.”

And I’m not even going to point out that because of high earner distribution, over 50% of everyone earn less than the national average income – so actually the fatties are doing well.

The Times yesterday reported that 42% of Grandparents never see their grandchildren again after a divorce or separation. But assuming there is no reason why the parents of the parents with custody should be so affected (about 50% of grandparents), that would mean almost every other grandparent lost contact. This seems unlikely.

I hope that the report producers: Families Need Fathers, the Grandparents’ Association and the Family Matters Institute aren’t including dead grandparents in order to create a more exciting number. I think that would be cheating. The Daily Telegraph has the same number but mentions that the sample size is a whopping 211. Were they perhaps recruited from the ranks of Families Need Fathers, the Grandparents’ Association and the Family Matters Institute?

Crapstats Rating: 1. Honesty score: 0

I haven’t commented on the torrent of data pouring onto our screens about the credit crunch because financial journalists do seem more numerically aware than social science commentators. And if I could tell which parts were crap I’d be rich rich rich and my butler woud be writing this.

Meanwhile this blog tries to keep to its purpose of discussing the merits of numerical data. However this is too good to miss – not so much crapstats as mind expanding.
The folowing was given to Blog Junior at school as part of a talk about body image from a recovering atman.

Where to start? The reference is to a paper written in 1971: did they really make sure that the first spider had received a nourishing dish of scrambled flies on toasted flies before they took the photo?

“Speed and Sugar” = “coffee & high sugar cereal”. Dream on.

My spidey sense tells me that self esteem is only a problem for spiders in their alter ego – and Steve Ditko agrees with me.

I remember the grapevine about this study at the time – some say that there was another photo of a spider having a trip. It had spun no web at all! Some said this showed a breakthrough in its consciousness as it stepped out of “the man’s” trap. Others suggested that it had stepped out of the trap game with fatal consequences.

On the other hand its self esteem was rocketing because it had just listened to LA Woman 10 times it a row: “is the record player on, like, you know, repeat, man?”

Today. the Times published an article on the large Hadron collider which contained a series of such bizarre comprehension-lite comparisons (sadly none with OSSPs) than I can only assume that it was deliberate. So its (ten gallon) hats off to Mark Henderson, Science Editor. Read it here.

Pick of the bunch (but sadly not on the website version):
“140,000 fridges full of sausages could be kept at a temperature colder than outer space by the magnet cooling equipment.”
“2 British Libraries could be filled with the data the LHC will generate every year.”

In the paper there is a really cute little graphic of a London bus to scale against a picture of the ATLAS component, whose purpose is to ’search for extra dimensions, dairk matter and the Higgs boson.”

There’s a great throwaway comment about the extra dimensions: “some theoretical physicists suggest that there could be as many as 26. Most physicists find these every bit as hard to visualise as normal people,” I’d like to meet the others but I’m not sure they could see me.

There’s an unpleasant insight into a physicist who’s really passionate about his work: “That’ll be the first sight of relief, that there are no obstacles in the vacuum chamber,” Dr Evans said. “There could be a Kleenex in the chamber – we’ve had that before. Only when we get the beam around will we be able to tell it’s clear.”

Finally, a bizarre comparison which Matty points out is also a crapstat: “The two streams will collide, at four points, with the energy of two aircraft carriers sailing into each other at 11 knots, inside detectors so vast that one is housed in a cavern that could enclose the nave of Westminster Abbey.” They don’t convert that vastness into Olympic sized swimming pools – but that’s not the main point, its about the energy. Firstly energy is measured in Hiroshimas or Suns, although this may be too little energy for that. Secondly the energy of two aircraft carriers sailing into each other at 11 knots is in fact the same as the energy of two aircraft carriers sailing parallel to each other at 11 knots; and roughly the same as one aircraft carrier sailing alone at 22 knots. Indeed it is roughly comparable to the energy of a London Bus travelling at 147,000 mph.

In my physical copy of the Times yesterday, it discussed a recent headline in the Daily Express promising 10 days rainfall in a single day last Friday – and invited readers to submit similar examples of twaddle to
badstats@thetimes.co.uk

Note that there is no reference on the website to this – and my email hasn’t got through to them yet either.
So now you know where to send the stuff when you find it, but my question is haven’t they already got enough themselves?

This blog has been lying low, keeping its head above the water but below the parapit – til forced out by today’s issue of the Times. All the usual problems:

1) Lack of sub-editing:
A £20 pound wind turbine made from reycled materials… can produce enough electricity to run lights for 63 hours or a radio for 30 hours and be built from scrap by unskilled workers in a day.

2) Lack of maths
News environment (where else?): “The average person throws away four times their own body weight in food each year“.
How much food is thrown away each year in UK: source same same article:
By private households: 6.7 million tonns. All sources: 18.4 million tonnes.
Population of UK (models own estimate): 60 million.
Average weight of average person in UK if the above refers to private households: 28 kilos – less than 4 1/2 stone per person! If it refers to all food waste – then it is hardly thrown away by the average person any more than the average person in the UK has killed 1/250th of an Iraqi since the invasion.

3) Lack of sense:
One of the largest and best preserved Roman villas yet discovered in Britain has been unearthed by archaeologists. Built 1800 years ago on the isle of Wight, the building is as vast as an Olympic swimming pool and shaped like a church.
“But what is wrong with that statement?” asks one of our younger readers. “Simple,” your blogmeister replies, “it should read Olympic sized swimming pool.”

We’ll come back to the question of food thrown away in future postings: does the average person really throw away 100 grapes every year?

“Health Secretary Alan Johnson said at the weekend that the obesity epidemic could lead to a public health crisis on the “scale of climate change”.”

What did he mean? where? for whom? How can these things be compared? If the planet literally become too hot for life, or are we talking about the odd outbreak of malaria?
Of one thing however I am sure: its healthier to be overweight than underwater.

Next Page »