Skip to main content

Lies, Damn Lies and Statistics

  • Gas is cheap.
  • The job market is strong.
  • Consumer debt is down.
  • The nation’s weather is cold.
  •  “Retail sales are expected to top $22 trillion this year, including $1.3 trillion in e-commerce led by China and the U.S.”  - eMarketer
  •  “U.S. Retailers likely to just meet holiday forecasts.” - Reuters
  •  “Total retail sales tumbled 11 percent to $50.9 billion during the extended Black Friday weekend. “ - National Retail Federation (NRF).
  •  “Online sales on Thanksgiving were up 14.3 percent, while Black Friday online sales were up 9.5 percent.” - IBM
  •  “Despite growth, Black Friday has lost its punch.” - Retention Science

  • The NRF data above is based upon a consumer survey, not purchasing data. 
  • The IBM data is from eCommerce “shopping cart” transactions. 
  • Cold Weather is good for in-store sales - It brings consumers into stores to buy outerwear and accessories,
  • Cold weather is bad for in-store retail sales - It keeps consumers at home, where they shop online and have fewer “impulse” purchases.
To paraphrase Andrew Lang;
Retail analysts use statistics in the same way that a drunk uses lamp-posts—for support rather than illumination.” 

Statistical Disconnects: Super Cats & Deadly Ice Cream 

While media, retailers, wholesalers, politicians, economists and analysts may misinterpret statistics, there is a more significant problem when the data used is itself inaccurate.
Many statistical studies are poorly designed.  
The collection, or analysis of data may not match the purpose of the study.   
 IQ testing is one classic example of mismatched analysis.  The test does not measure intelligence; its stated purpose. IQ tests do not measure problem solving, creativity or emotional intelligence. And, IQ tests are culturally and racially biased; they use “context” related questions such as, “Where does milk come from?”  Is the correct answer “a cow,” “a supermarket,” or ""? IQ tests may be effective at intelligence bench-marking and predictive academic achievement. 
 “The Pepsi Challenge” data illustrated that customers preferred the taste of Pepsi over that of coke. There were two “fallacies” in the study. First, the study used “sips” from small paper cups; sweeter taste is preferred in small sips, but not in larger consumption.  Second, a possibly more “valuable” data point was not shared; consumers who “discovered” through the survey that they preferred the taste of Pepsi, still preferred to buy Coke: “We’re a Coke household.”
“Cherry-picking” of data.  
Statisticians are skilled at choosing data to include in their study, and that which they disregard.  
Poor data sampling.
1n 2008, 132 cats were brought to Manhattan Animal Hospital, they had fallen out of open windows. Of the 132 cats, 128 survived, some from 30 story falls.  The New York Times wrote of the miraculous survival instincts of cats; the ability to “adjust rotation orientation” during a fall, “flying-squirrel” aerodynamics, joints and muscles acting as “shock absorbers.”  The media discussion lasted for months.  
The data point that ended the super-cat conversation regarded data sampling.  One woman interviewed said, “my poor cat must have been the exception. She fell out our 9th story window and died. Of course, I did not bring her body to the hospital, nor did I report it…”  The survey only included cats that had survived their falls.  When cats that died on impact were included the study it was no longer newsworthy. 
Presentation of data may bias interpretation.  
 “Milk with 3.5% Fat” - wow, that sounds like a lot vs. “96.5% Fat Free Milk” - better? This is the fat percentage for whole milk. Low fat milk is usually presented as “Reduced Fat, 2% Milk.”
Poor understanding of “probability.” 
There is no better presentation of the "probability" discussion than this piece, by Stephen J. Gould, "The Median Isn't the Message." Gould had been diagnosed with cancer (abdominal mesothelioma), he was informed that he had "a median mortality of eight months." (Note: Gould lived another, prolific, 20 years.)
Misunderstanding “correlation vs. causation.”  
Fact:  Children with larger feet score better on spelling tests. Interesting.  
Many observers have attempt to find “causality” in this data.  Why do children with larger feet spell better? 
Did A cause B? Larger feet cause spelling skills to improve. Is there a nutrition cause?  Something about shoes?  Foot size is likely genetic, is there a genetic component? Nature over nurture?  
Did B cause A?  Perhaps skill at spelling causes hormonal changes? Could time spent on academic achievement increase hunger, food consumption and therefor foot size?
Another view at the data illustrates that children’s feet grow larger as they grow older.  Overall, 8-year-olds have larger feet than 5-year-olds, and 15-year-olds have larger feet than 8-year olds.  Older children tend to spell better than than younger counterparts.  Foot size and spelling are correlated - but one does not cause the other.

Fact: As ice-cream consumption increases, so does the local incidence of drowning.
Ice-cream does not cause drowning. Nor do drowning incidents cause local residents to eat more ice-cream.  As summer temperatures rise people buy more ice-cream; they also spend more time in the pools and the ocean.  

Is More Better?
We have more data and statistics available than ever before, and it’s moving beyond our comprehension. Big data, by definition, is so complex that traditional data processing can not capture, filter, or process it. 
The real question:  What to do with all this data?
We can, effectively argue that warm jackets and mittens sell better in cold weather.  These statistics are not only correlated, we can likely show causality.  This season’s cold weather yielded high sales levels for warm apparel and accessories.  The weather, possibly, brought more customers into stores to purchase these items (weather that is too cold may keep customers home, or induce them to shop online.) While in stores, these customers may have purchased items they would not have purchased otherwise.
As we cannot, yet, control the weather and the manufacturing and purchasing of cold weather apparel is made 9 months ahead of the winter season… how are we to use this year’s weather data to make better purchasing decisions next year?  
Regardless of the weather next year, some cold weather apparel will be sold. Study the choices consumers made this year, the competitive environment, visual and technical trends, and related statistics.  Build great products and present them in a compelling fashion.  Sometimes, you just have to outperform the other guy.

Statistics: to use them is perilous - to not use them is ignorant.  
For 2015, let's resolve to... ask smarter questions, design better studies, challenge results, study probability, understand correlation & causation, and weigh the risks and rewards of acting on statistical data and analysis.

Lies, damn lies, and statistics. In 2015: It's our job to tame, understand, and take advantage of this Mark Twain attribution.

Happy New Year.

(c) David J. Katz - Detroit, Michigan - December 28, 2014

Popular posts from this blog

Beware of Wombats & Other Vampires

You are surrounded by dangerous WOMBATS. They’re everywhere. Sometimes they hide in plain sight, easy to spot. Other times they are well camouflaged, requiring heightened awareness to identify them. You need to stay alert, it’s important to avoid them. WOMBATs resemble ordinary, productive tasks. However, they are vampires for time and resources, weapons of mass distraction.WOMBATs are seductive. Working on a WOMBAT feels productive.WOMBATs are bad for your career.WOMBATs are bad for your business.WOMBATs infiltrate your work day (and your personal time). Strike them down.WOMBATs may be be ingrained in your company culture: “We’ve always done it that way…” WOMBAT Metamorphosis Alert: A task or project that wasproductive in the pastcanevolve into a WOMBAT in today's environment.Your comfort zone is populated with WOMBATs.More on comfort zones, here.Some people are WOMBATs in disguise. Stay away from them, they are vampire WOMBATs.If you don’t control your WOMBATs, your WOMBATs will…

Taking Tips From a Younger Generation

Phyllis Korkki, an assignment editor at The New York Times, visited the garment district in Manhattan to interview designers as part of a story for the newspaper’s Snapchat account. Credit George Etheredge/The New York Times
What Could I Possibly Learn From A Mentor Half My Age? Plenty.

How on earth did I become an “older worker?”

It was only a few years ago, it seems, that I set out to climb the ladder in my chosen field. That field happens to be journalism, but it shares many attributes with countless other workplaces. For instance, back when I was one of the youngest people in the room, I was helped by experienced elders who taught me the ropes.

Now, shockingly, I’m one of the elders. And I’ve watched my industry undergo significant change. That’s why I recently went searching for a young mentor — yes, a younger colleague to mentor me.

How Randa and the Fashion Industry are Adapting to DIY

The term 'Do It Yourself' has turned into a phenomenon over the past decade and is continuing to gain momentum, especially in the fashion industry. From interactive design stations at Topshop, to custom shoes at Jimmy Choo, every level of the fashion industry is dipping their toes into the pools of DIY.

"Many industry insiders think it is just the beginning. Ask about the future of fashion, and the answer that is likely to come back (along with the importance of Instagram and the transformation of shows into entertainment) is personalization," says Vanessa Friedman from the New York Times.