Sunday, February 28, 2010

Signal Fails in Attempt to Use, Understand Basic Arithmetic

When not preoccupied with accepting plagiarized submissions or patting its own back, The Signal’s editorial board finds time to show off its dazzling statistical incompetence. TMS’s Sunday editorial[1] is the most recent example. The piece discussed changes in the amount of crime in Santa Clarita between 2008 and 2009, and in doing so implicitly defends three ridiculous conclusions: (1)The non-violent crime of theft is just as important as the violent crimes of rape and murder when calculating our city's crime rates, (2)Year-to-year changes in crime rates are meaningful, and (3)There is no further need to dig into the numbers provided by SCV Sheriff’s Department in order to understand crime in Santa Clarita.

The First Dumb Conclusion: All Crimes are Equal

Our justice system recognizes crimes as being unequal in severity—that’s why you fry in the electric chair for committing first-degree murder while you just get a monetary fine for speeding. This distinction was lost on The Signal’s editorial board members.

In their article, they say “Crime was down by 1.1 percent in the city of Santa Clarita.” The 1.1% value is a meaningless, aggregate statistic that counts larceny the same as it counts rape, that mathematically weights auto-theft as being as important as murder. It was arrived at by adding up all the crimes committed in 2008, all the crimes committed in 2009, and looking at the percent difference.

The problems with this approach are many. But the central problem is that the big statistic used to describe Santa Clarita's overall crime rate means that whether you’ve prevented 10 car thefts or 10 murders, you’ve improved the City’s crime rate by the exact same amount. Of course, we’d all rather live in a city with ten more car thefts per year instead of ten additional murders, but no matter. A crime, whether violent or non-violent, counts exactly the same.

What really happened between 2008 and 2009[2]?

*There were 2 more homicides in 2009 than 2008
*There was 1 more rape in 2009 than 2008
*There was 1 more robbery in 2009 than 2008
*There were 4 more aggravated assaults in 2009 than 2008
*There were 69 more burglaries in 2009 than 2008
*There were 4 more larceny thefts in 2009 than 2008
*There were 13 more incidents of arson in 2009 than 2008
*However, there were 134 fewer auto thefts in 2009 than 2008

Indeed, the one-and-only factor responsible for the claim of a 1.1% decrease in crime in Santa Clarita is a large decrease in the number of vehicular thefts. However, The Signal’s editorial team chose to not to distinguish between types of crime, instead writing rather generically and generously that “Crime was down 1.1 percent in the city of Santa Clarita.” That’s a deliberate misinterpretation of numbers. There were more homicides, rapes, robberies, aggravated assaults, larcenies, and acts of arson—but on the bright side, fewer cars got stolen. The paper presented these values in an earlier story, but for those who read TMS only on the weekend, this vital decomposition of the data into its constituent parts was missing. If honesty was a priority, the editorial should have read “Seven out of eight classes of crime increased by a few incidents per year between 2008 and 2009, but there was one exception—fewer vehicular thefts occurred in 2009.” Instead, TMS seems not to care whether it’s one less homicide or one less car theft—any reduction in any crime counts the same.

The Second Dumb Conclusion: Year-to-Year Changes in Crime are Meaningful

They’re not. You need far more data points to establish a statistically significant trend[3] in crime reduction, and there are these things called “noise” or “error” that mean random events can cause observed values to diverge from a characteristic trend.

Put simply, no one should be using the ridicuolous 1.1% value or any other single number to suggest how crime in Santa Clarita is changing. But that’s exactly what TMS did. The Signal then shameless fawns over the incumbents, saying they’re “taking a bite out of crime.” Note to The Signal: a 1% change in a meaningless statistic does not a “bite out of crime” make. At least their out-dated cliche drew attention away from their fauly reasoning.

The Third Dumb Conclusion: The Numbers from the Sheriff are Sufficiently Informative

The editorial on crime in Santa Clarita simply parroted numbers provided by the Sheriff's Department. However, the writers could have made these numbers much more interesting by answering a few simple questions. For example:

1.Controlling for population growth, is crime increasing or decreasing?
2. Is there a higher per capita rate of crime inside or outside city boundaries?
3. Are there long-term, statistically meaningful trends in how crime is changing in Santa Clarita?

Instead of pondering what the numbers really mean, the editorial bases its arguments off one really bad, non-controlled, uninteresting statistic. They then take the very embarrassing step of asking readers to ignore the headline they first used to describe crime in Santa Clarita:

"In the near future, you will open your mailbox and there, screaming at you from an obnoxious piece of junk mail, will be Tuesday's Signal headline, 'Crime on the rise in SCV,' complete with the subtext: "2009 saw increase in reports of several serious crimes.' [...] It will be a lie." That's right, the editorial board is saying you shouldn't trust its headlines as they're misleading (even when they're more factual than editorials.) How lucky we are to have such intelligent folks interpreting life in Santa Clarita for us all...and also telling people how to vote.

IHSCV Conclusions:
*The Signal's editorial was an embarrassment that demonstrated no ability to critically evaluate statistics.
*No one, whether Ken Pulskamp, TMS, or the unnamed CC candidate responsible for the mailer should use statistics on annual changes in crime as evidence for trends or accomplishments; they mean very little, except when you can supply other evidence to link cause and effect (ex: the Sheriff's Department's aggressive pursuit of gangs and a corresponding drop in gang-related crimes).
*It's clear that TMS is endorsing the incumbents by lavishing them with undeserved praise--what a surprise. Way to go, editorial board!

[1]Here's the link--I can see why they don't sign editorials such as these.
[2]Here's the source, the SCV Sheriff
[3]Statistical significance has a very precise meaning, which is also lost on TMS. They don't quite grasp that a small change in a value from 2008 to 2009 says very little about the shape of the underlying trend.


Anonymous said...

The incumbents need to do the decent thing and pay the Signal for the 1/2 page ad.

Anonymous said...

that is a great point about the incumbents. Thank you Iheart scv for enlightening as to the statistics.

I would like to know who ACTUALLY WROTE THAT PIECE. Trace it back and it probably came from one of the incumbents' campaigns.