Yesterday, the insurance sales website Insure.comreleased a poorly constructed and remarkably misleading study on auto insurance rates.  (More surprising, perhaps, is the fact that some journalists uncritically posted the study as news.)

The study purports to organize states according to how expensive it is to buy auto insurance in those states.  The results place Louisiana as the most expensive place to buy car insurance and Maine as the least.  And while premiums in Louisiana are quite a bit higher than average and those in Maine lower, the methodology used by Insure.com was so insufficient that the only thing you can really take from the study is that driving Mercedes vehicles can be very expensive no matter where you live.

The study places Georgia, Oklahoma, Montana, California and Kentucky among the top 10 most expensive places, even though drivers in each of those states spend less each year on auto insurance than the national average (according to the much more comprehensive data assembled by the National Association of Insurance Commissioners).  Coming in at an apparently low-priced #43 is the state of Washington, despite the fact that drivers there spend, on average, about 24% more on auto insurance every year than drivers in Montana (#6), 16% more than Oklahoma (#4), and 9% more than Georgia and California (#3, 7).  Drivers in perennially high-priced Nevada must be confused by the #36 ranking given by Insure.com, seeing as how the average Nevadan spends 17% more on auto insurance than the average American.

So why did Insure.com get it so wrong? They apparently spent all their resources varying the study by model of car rather than trying to get a real picture of the rating systems used by various companies in various states.

Insure.com researched premiums for 750 models, including 37 different types of Mercedes and 18 different Porsche models, which probably gives a very good sense of whether or not your car will be more or less expensive to insure no matter where you are. 

However:

• They only reviewed six different companies, which represent about 62% of the national marketplace.  Aside from ignoring 40% of the countrywide marketplace, this formula is particularly useless in states with highly competitive markets like California where these six companies represent only about 35% of the market.  There are about 160 other insurers in California selling 65% of the policies in the state.
• They only reviewed premiums in 10 ZIP codes in each state. Because ZIP code plays such a substantial role in premium setting around the country, selecting ten can be very misleading.  Even in California, where the use of ZIP code is more limited than in any other state, choosing only ten ZIPs likely produces a biased sample.  In Maine, which Insure.com calls the cheapest state, there are about 500 ZIP codes.  Michigan, ranked as the second most expensive state, has nearly 1,200 ZIP codes.  California has about 2,700 codes. As a result, it is quite possible that the selection of a different ten ZIP codes in each state would markedly change the rankings.
• Including credit score in the testing skews the results.  Insure.com tested rates assuming that the driver maintained a good credit score.  Most states, with the notable exception of California, Massachusetts and Hawaii, allow insurance companies to use credit scores in rate setting and it is playing a prominent role in pricing. Somewhere between about 45% and 55% of Americans have less than “good” (>700) credit scores, so about half of Americans will not qualify for the rates listed by Insure.com and will instead pay significantly higher rates.  This also negatively impacts the rankings of the three states that prohibit the use of credit scores. For most states, the study cherry-picks low-priced policies (only those offered to drivers with good credit scores) while not exclusively testing low-priced policies in California, Massachusetts or Hawaii.
• The methodology likely includes how long a driver was insured with any carrier in the past.  Insure.com does not indicate whether the researchers tested customers who have been insured in the past or not or how long they’d been insured.  I’m guessing that they tested people who had been insured for at least five years and probably 10. (I will happily correct this if their researchers clarify the methodology.)  Like credit-scores, this factor biases the results towards lower rates than will be seen by many millions of drivers who had a lapse in coverage, served abroad in the military, used public transit for a time or simply were not driving at some point in the past several years.  This particularly impacts California’s ranking, because that state prohibits the use of prior insurance as a rating factor.  This means, again, the results for California are not cherry-picked from drivers with the cheapest characteristics, whereas other states samples are biased downward.

The net result is that Insure.com has issued a study that does not accurately reflect the cost of insurance that drivers around the country will likely pay, either in actual dollars or relative to drivers in other states.  The study is a reminder that if you happen to drive a luxury car worth more than $100,000 you will pay a lot for auto insurance, but it’s not much else.

p.s. The Center for Economic Justice's Birny Birnbaum, an economist and former Associate Commissioner at the Texas Dep't of Insurance, notes one thing that is illustrated by the Insure.com survey: "The study points out how opaque auto insurance pricing is in most states — CA being a notable exception!"