BBC Newsbeat and use of dodgy statistics

This entry was posted by on Tuesday, 31 January, 2012 at

I made a complaint to the BBC last week about an article they were running on blackberry users being unsatisfied. I'm not currently a blackberry user satisfied or otherwise, I'm not a RIM employee/shareholder or anything to do with the company – I just found the blatantly bad use of statistics really, really annoying. 

"I found the article at http://www.bbc.co.uk/newsbeat/16740008 to be very misleading, the article states that: "Newsbeat's been in touch with hundreds of users and nearly three-quarters say they want to change their handsets." On the radio show, this was changed to "hundreds of users have contacted us". This has been compressed into a headline used on your site as: "Three-quarters of BlackBerry users want to switch phones". This is deeply inaccurate, it may be the case that 71% of the "hundreds" of people that contacted newsbeat were unhappy with their blackberries but to then scale this up to a %'age of all blackberry users is wrong. I also fail to see how this is a story, the figures themselves show that there is only a small drop in sales(100,000 per quarter). I look forward to hearing back from you."

I got a response from them yesterday:

Many thanks for getting in touch. We take comments and complaints about our output seriously so your feedback is welcome. You're absolutely right to say the headline on this story was misleading – we should not have extrapolated the 'three-quarters' figure and applied it to the entire population of Blackberry users. As a result of this, and some of the other issues you talk about in the statistics, I have reminded the entire team of the importance of fully understanding how to apply research statistics to stories. As to your further point about sales figures, we have run a follow up piece to the item today, in which we have reported a further set of sales figures which apply to Blackberry which the company itself claims shows sales are holding up.

The problem with bad statistics like this is that they stick – the first page of search results show that this has been discussed in a number of forums and repeated on other news/tech sites. We should expect more of the BBC – they should all read @bengoldacre's book Bad Science to understand more about dodgy stats and the dangers that come with them.

The updated story mentioned is here and they have updated the story here. The original article has been archived on another site here and I've taken a screenshot of it here in case that disappears as well.