Some fun quotations . . .
- “42.7% of all statistics are made up on the spot.” — Steven Wright
- “There are three kinds of lies: lies, damn lies, and statistics” — Benjamin Disraeli
- “Figures don’t lie, liars figure”– Mark Twain
- “There are two kinds of statistics, the kind you look up, and the kind you make up.” — Rex Stout
I’ve been doing technology marketing for over 20 years. One of the hardest things for marketing teams to do is to come up with credible ‘proof points’ to help justify the hard and soft benefits of their solutions. A proof point is supposedly a fact-based statement backed up by a tangible reference. A classic example would be “Our customers reduced their operational costs by 8% in the first year and achieved a 213% ROI in less than two years”. The challenge is that most marketing teams have never built valid case studies that could give rise to real proof points. The teams often lack the financial literacy to actually calculate the proof points and they rarely can get access to or permission from real customers to do this type of analysis. When push comes to shove from the sales team, marketing organizations often just make up proof points and hope that no one ever challenges their validity.
The Wall Street Journal’s ‘Numbers Guy’, Carl Bialik, just published an article entitled ‘Marriage-Maker Claims Are Tied in Knots. The subtitle of the article is ‘Online Dating Sites Say Hordes of People Ultimately Marry, but Their Methods Have Plenty of Hitches of Their Own.’ In the piece Carl talks about the bogus nature of online dating sites claims of the number of marriages that have occurred as a result of people hooking up on the site.
“Online-dating sites have changed romance for millions of Americans. But claims that such dating leads to hordes of newly wedded couples may be fairy tales.
EHarmony claims in television or online ads in the U.K., U.S. and Australia that 2% of Americans who got married last year met through its site. But the stat is based on an online survey. Similarly, a Match.com media kit claimed that 12 marriages a day trace their roots to the site, but the company now says it’s inaccurate. And Markus Frind, chief executive and founder of Plenty of Fish, doesn’t advertise about marriages, but says his site brings about 100,000 marriages a year, a figure based in part on “some study I found online.
. . . Mr. Frind, of Plenty of Fish.com, scoffed at the dozen Match.com nuptials per day, noting that eHarmony claims 10 times that number. He says that his site creates 800,000 relationships each year, according to exit interviews with departing members. He says that works out to about 100,000 marriages per year, based on a study, the details of which he couldn’t recall, that says 10% to 15% of relationships lead to marriage.
The only statistic Mr. Frind knows with certainty, he says, is the number of members who have self-reported success stories on his site — now around 2,000. “I don’t want to pay $200,000 to a research company to find out how many marriages I have per year.”
It’s clear from Carl’s reporting that the dating sites basically made up their claims and are not too interested in investing the time and money to find out what the real numbers are. I was surprised to learn that there is a lot of formal research on this topic. Terms such as Ipse-Dixitism and Argumentum ad Verecundiam have been used to describe this syndrome before. Argumentum ad Verecundiam or argument from authority is defined in Wikipedia as “Argument from authority or appeal to authority is a logical fallacy, where it is argued that a statement is correct because the statement is made by a person or source that is commonly regarded as authoritative.”
The most common tactic employed by technology marketing organizations to overcome a deficit of proof points is to hire an industry analyst firm like Aberdeen Group, Forrester, or AMR Research. These firms all provide consulting services to technology providers to help them research, position, and prove the effectiveness of their technology solutions. Typical services include sponsored white papers, research studies, webinars, or speaking engagements. In general, these firms do a very good job at providing these types of services. During the height of the Internet Bubble some analyst firms were accused of being paid shills for some technology providers. While there may have been some truth to certain allegations the industry as a whole has significantly cleaned up their acts since then. Most notable is the Gartner Group. Gartner created the Office of the Ombudsman whose mission is “to openly and assertively address issues of analytical independence, accuracy and integrity through compliance with Gartner’s Principles of Ethical Conduct.”
The primary challenge with working with industry analyst firms is cost. A single project can cost anywhere from $25,000 to $50,000. Analysts and consultants conduct dozens of engagement in a year – they have a very wide and deep experience base. Since they’ve worked with so many technology companies they have a pretty good idea about what works and what doesn’t work. In today’s tough economic times a lot of marketing teams cannot afford to make single project investments of this scale. If your company is in this kind of situation I have a plan that you could execute on your own to build and leverage credible proof points for your solutions.
The core of this project is helping customers document and then promote their own success using your solutions. Everyone likes to have their horn tooted. By helping your key users or champions demonstrate the wisdom of their decision to acquire and deploy your solution you can create a win-win scenario. For lack of a better term I’ll call this a ‘Customer Success Program’. The goal of the program is to get 6 to 12 customers to participate in a study to document the business impact your product or service has had on their organization. The deliverables of the program will be individual case studies and a micro site or blog. While the customers who participate in the program can distribute the case study internally, as a vendor you will anonymize their identity in any external presentation of the information and only use summary information. This approach has two benefits. First, by creating a solid, fact-based analysis of the customer’s success you enable the key users and champions to tout their success inside of their company. By anonymzing their identities and using only summary information externally you can typically get over any of the typical hurdles customers have with public endorsements of your products and solutions.
Your real goal in this project is to develop 3 to 5 rock solid proof points that you can make about your products and solutions. To make this work you’ll probably need to approach a pool of 30 to 40 customers, split up into three to four verticals. If you can get 6 to 10 customers to participate, you will have hit a home run. Also, you’ll need to identify inside of your organization whom has the best personal relationship with the target customers and users – sales, professional services, customer services, etc. You will need to develop a basic survey instrument – situation assessment, business problem, decision making criteria, hard and soft benefit categories, risks and risk mitigation strategies. Leveraging your firm’s best personal contacts, ask for a brief teleconference with your targeted users/champions to explain the program and ask for participation. Emphasize that this is a fact finding mission on how effective your product or service has been and that you are interested in learning how you can improve your offerings even further. Stress the anonymity in the use of any results from a marketing and sales perspective and emphasize the benefits of having a documented case study for internal consumption in the customer organization.
You will learn a lot of things in the execution of this kind of project. First you will find out who inside of your company really does have good relationships with key people at your customer organizations. Second, you’ll learn that a lot of people are passionate about your technology and more than happy to share their experiences. Third, it is inevitable that you will learn somethings about your product, services, and company that you did not want to hear but definitely need to be fixed. Finally, you will be able to harvest real fact-based information to support the proof points you need to effectively market and sell your solutions.
While it may be easier just to make up some marketing statistics and claim victory, the process of actually talking with your customers and documenting their success will actually make your company successful.
