Earlier this week, Nate Elliot, a vice president and principal analyst at Forrester (a respected research firm), wrote an open letter to Facebook’s CEO Mark Zuckerberg.
Facebook is failing marketers.
I know this statement sounds remarkable, perhaps even unbelievable. After all, you offer marketers access to the largest audience in media history and you know a remarkable amount about each of your users. As a result nearly every large company now markets on Facebook. Last year your company collected more than $4 billion in advertising revenues.
But while lots of marketers spend lots of money on Facebook today, relatively few find success. In August, Forrester surveyed 395 marketers and eBusiness executives at large companies across the US, Canada and the UK — and these executives told us that Facebook creates less business value than any other digital marketing opportunity.
According to Forrester, Facebook came in last out of 13 online marketing strategies and sites.
Elliot’s criticisms were met with a strong response from Facebook and other media outlets. Facebook replied:
the conclusions in this report are at times illogical and at others irresponsible. The reality is that Facebook advertising works. That’s why we have more than a million active advertisers including all of the Ad Age100.
Facebook is right to question the survey methodology in the Forrester report. Too much for what is passed-off as “research” today is poorly developed, poorly defined, and poorly executed.
The Forrester report is materially flawed. Among other things, Forrester surveyed a small group and used a screened sample that although presented as random, was not random at all. In fact, Forrester admitted this when it accompanied its report with the following statement:
This data is not guaranteed to be representative of the population, and, unless otherwise noted, statistical data is intended to be used for descriptive and not inferential purposes.
Other media outlets reacted quickly. For example, Business Insider argued that:
In the unfortunate all-too-common-lately style of high-level broad spurious insights without any detailed methodology, most of the actual ‘data’ is only available behind a paywall, but there was enough publicly available to see some of the major issues with it and then the benefit of reading the report compelled me to speak out here.
There are 13 types of marketing Forrester compares in the chart BI shared and, unfortunately, with a spread in marketer satisfaction of just 8 percent from lowest to highest, the differences are likely not statistically significant. I can’t actually tell for sure since the sample is at most 395 “executives” for the question but likely is lower per response, since not all of these executives have likely tried all 13 of the marketing types (Google+ marketing anyone? Hello, who are you?!). From data in the same report it appears less than 50% of the sample utilizes Facebook paid advertising, for example, calling the significance of an 8 percent difference even further into question. Having a separate category for “mobile marketing” too, ignores the fact that Facebook is the world’s most popular mobile application /advertising platform by some margin.
I respect Nate Elliott but think he’s hyperventilating a bit here. The study he sites scored marketer satisfaction on a scale of 1 to 5. Facebook got a 3.54 vs. 3.83 for paid search, just above moderately satisfied. Put another way, marketers like Facebook 7.6% less than Google. The real headline should be “Marketers Moderately Happy with Facebook Ads, But Just a Bit Less Than Google.” I wouldn’t extrapolate too much from that, Nate.
People were right to question the efficacy of Forrester’s research, but they missed the forest from the trees.
Whether marketers like Facebook significantly less than other marketing channels and sites, or insignificantly less, begs the question. When you look past the link-bait and sloppy research, beyond the anger in Facebook’s response, and beyond the noise from the talking heads, the question Forrester should have asked is:
Does Facebook marketing actually work (particularly for small businesses)?
Sure, people will offer data on increased clicks, visits, impressions, etc. But those are all soft metrics. You can get a 13 year-old to click on any ad just by varying the image on that ad. It’s not enough to know that Facebook creates less value than other marketing channels. Does it create ANY meaningful value?
Businesses are spending billions of dollars marketing products and services on Facebook. Marketers are making tons of money. Facebook is making tons of money. It’s a win-win – except for the vast majority of the million advertisers burning through their budgets on ineffective Facebook ads.
Marketers and analysts are fiddling while marketing budgets burn. Advertising is broken.
Design Done Better
The easiest way to get affordable, high-quality custom logos, print design, web design and naming for your business.Learn How to Grow Your Business With Beautiful Design