I was wondering the other day whether statistics have any real value other than providing points of arguing. For instance you could claim that the government should invest more in X than Y since your survey says that twice as many people like X than Y, case proven! I have already blogged about the problem with the lack of ability to say "no" in surveys. In the example of the Olympics, saying that 500,000 people signed a petition asking for the Olympics is impressive unless you have the chance to report that 45 million didn't want it.
Another thing that has come up recently however is the weakness of survey questions. An article I read on the BBC news site was talking about how the "don't knows" might not be people who don't care as much as people who can't answer the question because they are not sure quite what it means.
Consider the question, "Do you regard yourself as belonging to any particular religion?" (taken from the National Centre for Social Research). A typical type of survey question whose data will end up somewhere important. But the answer is not necessarily straight-forward. If you are an atheist, the answer is simple, if you are Muslim or even Catholic, the answer is probably also straight-forward. But think about a bog-standard Christian. Firstly I don't call my faith a religion because religion to me is not synonymous with faith and is not what I practice (even though I call myself a Christian). Also, even if I do consider my Christianity a religion or understand that the question probably does not make the distinction that I do, do I belong to a particular religion? How particular? I am a Christian but do not call myself Anglican or Evangelical so should I answer "No" because I am not particular?
The problem for statistics involving large numbers of people is that if 30% of Christians say, "No" to the above question, the figures might suggest that the number of Christians is only 70% of the actual figure and then this figure is used to derive all sorts of "scientific" conclusions.
No doubt some people are supposed to be experts in these things but I would suggest any self-respecting information expert reject attempting to make any conclusions based on questions that are hard to decipher or at least provide a box labelled, "I am not sure I understand the question" which allows us to distinguish "I don't care" from "I'm not sure".
Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts
Wednesday, 15 September 2010
Monday, 30 March 2009
More insanity in the made-up numbers department
I was just reading this about people claiming the amount of global cybercrime outstripped the international drug trade by $1T against $400Bn. As the reporter pointed out, do they expect us to believe that the global cybercrime trade is higher than Saudi Arabias annual domestic product of $555Bn from 2007. As if.
What particularly troubled me was the comment by security firm Finjan about justifying the touting of these numbers: "In our Q1 2009 report on cybercrime, for example, we revealed that one single rogueware network are raking in $10,800 a day, or $39.42 million a year," it said. "If you extrapolate those figures across the many thousands of cybercrime operations that exist on the Internet at any given time, the results easily reach a trillion dollars."
Can anyone else see the problem here? These people are supposed to be experts and not even their statistics make any sense. I am not a statistician or an economist but let us expose the error in their logic.
1) The example they used of a company taking in $11K per day would not have been making this every day for a year. They would more than likely either been tracked down and shut down or their source of revenue would have dried up as virus-checkers are updated or stolen credit card details are blocked.
2) This example of a company would be at the high end of cybercrime. Most people who write worms either don't make any money or certainly not as much as they might want and certainly not $11 per day. How many people would really make that much in a day amongst the 'many thousands of operations'? Not many!
So the evidence that was probably "company X made $11K in a day last year" becomes "every cybercrime company makes $11K every day of every year" it has been multiplied by 365 and then by thousands. To think these people probably get paid!!
What particularly troubled me was the comment by security firm Finjan about justifying the touting of these numbers: "In our Q1 2009 report on cybercrime, for example, we revealed that one single rogueware network are raking in $10,800 a day, or $39.42 million a year," it said. "If you extrapolate those figures across the many thousands of cybercrime operations that exist on the Internet at any given time, the results easily reach a trillion dollars."
Can anyone else see the problem here? These people are supposed to be experts and not even their statistics make any sense. I am not a statistician or an economist but let us expose the error in their logic.
1) The example they used of a company taking in $11K per day would not have been making this every day for a year. They would more than likely either been tracked down and shut down or their source of revenue would have dried up as virus-checkers are updated or stolen credit card details are blocked.
2) This example of a company would be at the high end of cybercrime. Most people who write worms either don't make any money or certainly not as much as they might want and certainly not $11 per day. How many people would really make that much in a day amongst the 'many thousands of operations'? Not many!
So the evidence that was probably "company X made $11K in a day last year" becomes "every cybercrime company makes $11K every day of every year" it has been multiplied by 365 and then by thousands. To think these people probably get paid!!
Subscribe to:
Posts (Atom)