I want to be upfront, this isn’t a hero’s tale.
It is not the usual story of hockey stick growth we are so often wooed by on Medium or GrowthHacker. This is what has to happen before that story can be told, or at least I certainly hope it is. In my first 90 days as the Head of Growth for Nutshell, I cut our Facebook spend from $20,000 a month to zero and the sky never fell.
It’s vital to note, before we climb into this rabbit hole, that I was certainly not alone in this decision. Make no mistake it was mine to own, but in many ways my entry into the company created an opportunity to give them cover to explore a hypothesis that had long been brewing.
Here’s how we got there.
Data, Data, and More Data—Where Do You Really Begin?
Philosophically, I’m a big believer in two things. First, successful digital strategies depend on a blended, multi-channel approach to engaging, capturing, nurturing, and converting potential users to customers. And second, the most successful strategies focus more attention on generating traffic and leads organically. When it comes to organic traffic you’re rarely paying for the same real estate twice. When it comes to paid acquisition you pay for the same real estate everyday and every day it gets more expensive.
Focusing on organic traffic and conversion sounds simple, but the addiction of paid acquisition is a powerful one (click the link and save it…great article). Gains are usually immediate and in an optimized spend, big wins are immediately reflected in the bottom line of your business. As a result, it’s just easier to obtain institutional buy-in for focus on paid tactics as opposed to the “long con” of organic growth.
With this in mind, your first priority as a new leader joining an established team, should always be assessing and optimizing your paid spend. Bloated accounts and inefficiencies chew budgets, throwing your whole program out of balance. I knew I would have to trim some fat, but from where?
You have to start by answering this seemingly simple question.
Is Paid Acquisition Actually Driving Growth?
In fairness, I was more than adequately warned. My predecessors and the super talented team at Nutshell had been trying to crack this code for a while. It was becoming increasingly difficult, if not impossible, to find any correlation between overall digital ad spend and new MRR. Spend would increase and decrease, but overall sales (new MRR) remained unaffected.
Note: There is a 14-day lag to consider in this data set. On average our trial users become customers in 12 days, but as you can see, even without the trend lines, there’s almost no rhyme or reason to the overall effect of paid acquisition.
I don’t believe (and neither should you) there should be a one-to-one ratio between spend and raw wins (number of new sales) or new MRR (dollar value of new sales), but a basic correlation on some scale should be apparent, i.e. you spend more you make more. With no correlation between our overall spend and sales growth, the next logical question to answer was…is the correlation being disrupted by a particular channel? Spoiler alert: It was not.
The only way to solve a problem is to make it smaller
So now what? No correlation in overall spend, none in individual channels. In my previous position as the strategy director of a small digital agency I saw hundreds of different data-related problems, but to be honest, nothing quite like this.
Like any large problem, your only option is to make it smaller, break it down into its requisite parts, and assess through a process of elimination. As a result I began by analyzing the performance of our two primary paid channels from bottom up.
Which Channels Are Driving Actual Sales?
We track lead source to sale using Nutshell and a little elbow grease from our engineering crew. (I may be a little biased, but I’ve used a lot of CRMs and Nutshell’s reporting capabilities are crazy powerful.)
As you can see the primary driver of new sales, according to our last-click attribution model is organic traffic. Specifically, branded organic traffic. However, because I’m a big believer in channel mix, last click is never the whole story. (Please ignore the “web signup” source, it’s used to differentiate between trials originating from our website vs. our mobile app and we’re not running any mobile app campaigns, so all trials are “web signup.”)
AdWords Tactical Execution
First, looking at the above chart you can see that our AdWords program is driving 30% of new sales. During the time of this analysis we were running very strong search campaign that bid on a healthy mix of branded (Nutshell), high-value terms (CRM, small business CRM, etc.), and competitor-focused keywords (Salesforce alternative). Alongside that program we were also running a very effective re-marketing campaign via the Google display network, driven primarily by the traffic generated from these and our Facebook campaigns.
The above chart makes a couple things clear.
- The Adwords program was an effective driver of customer acquisition.
- Sales from traffic and trials generated by Facebook advertising campaigns, considering what we were spending, were bizarrely absent.
Bottom line, AdWords is performing well.
Paid Facebook Tactical Execution—The Plot Thickens
Considering what our Nutshell data was telling us you would think we’d found the culprit. But nothing is ever that simple, right? When it came to Facebook we were running a pretty aggressive, and at times experimental campaign to attract free trial users.
Want to see just how experimental? Read this.
Brass tacks...Facebook CPC and CPM campaigns were driving trials. A significant number of them. That much was clear, but as you can see, Facebook-generated sales are nowhere to be found in our data.
Or were they?
For starters, the Facebook tracking pixel said they were. Designed to track any visitor who engaged a Facebook ad and purchased within 60 days, the pixel was consistently recording these conversions. Our own agency partners trusted the FB pixel and warned us that abandoning the Facebook strategy could have serious consequences.
Source of Truth
One of my favorite things about the Nutshell team, including our partners on the agency side, is that we are extremely data-driven and no one holds up sacred cows. Debate is vigorous and impassioned and I love it. So, whenever there is a data-based disagreement, as we had here, the rhetorical battle lines are almost always drawn over the veracity of our source of truth. In this case, was it Nutshell or Facebook.
Facebook Works. Our Attribution Data Is Flawed.
The argument is as simple as the section header above suggests. Facebook was working we just couldn’t see its success in our data because our attribution was flawed, the position held by our agency. Truth be told, we too had our questions about the accuracy of our attribution. We still do. Our agency pointed to strong overall sales while the Facebook campaigns were running, increases in traffic and leads from both organic and direct traffic sources, and what they thought was even under-reported data coming from the tracking pixel.
Furthermore, and to me the strongest part of their argument, Facebook was filling our re-marketing coffers with cheaper yet highly valuable traffic. Re-marketing was its own significant source of leads as you can see below.
As our Facebook spend increased, conversions from our re-marketing campaign increased in lock-step. Since we were utilizing isolated landing pages, there was no doubt that Facebook was driving this growth.
But…where are the sales?
Time to Go Deeper
Despite the rather convincing argument, I was still ready to pull the plug on Facebook for reasons you’ll discover in a moment. However, between our agency’s stern warning, the fact that I’d been on the job less than a month, and my steadfast rule that, like a surgeon, a strategist’s first duty is to do no harm, the prospect of 86’ing an effective lead generation program gave me serious pause.
It was at this time that I began to worry I looked less like a solid hire as the new Head of Growth and more like the Clark W. Griswold of digital marketing. (Warning: language NSFW.)
The Numbers Just Didn’t Add Up
Even before I stepped foot in the office, unbeknownst to the team, I was already diving into the data and discovering what I had many times before—that there were some significant disparities when it came to Facebook.
For starters, when I examined just the traffic driven to our website by Facebook, the tracking pixel over-reported website visitors by 25-35% vs. traffic in Google Analytics. Not a deal breaker, but a betrayal nonetheless and certainly not a strong argument to lean on sales attributed to Facebook as reported by the pixel.
And when I dove deeper, I expected to at least find some indication in Google Analytics that Facebook’s role in multi-channel conversions would be much more pronounced if the attribution model was changed to first click, as opposed to last click. This was not the case.
What Gives? I’ll Tell You…Facebook Sucks.
To be frank, I’ve got a lot of problems with Facebook. Save the fact that I blame it for the complete brutalization of social discourse, the channel as a paid acquisition tool has some serious flaws for B2B advertisers. Really, for any advertisers.
First of all, it presents itself as a viable alternative to paid search and it is not. Paid search is an intent-based channel, Facebook, on the other hand, isn't. While highly targetable, your Facebook advertising is at best a disruption, and at worst a downright rage-inducing violation of someone’s newsfeed sovereignty.
Finally, let’s face it, Facebook is a two-bit huckster. Whether we’re talking about data privacy violations or cooked reporting issues, the company’s penchant for duplicity is well-documented. I simply don’t trust the platform and frankly, neither should you.
Shut It Down for 45 Days
To me, the preponderance of evidence pointed in one direction: Facebook leads were not meeting our standard of quality and as a result, weren’t translating into sales. And with that, on April 30th of this year we suspended all Facebook advertising of any kind. And waited.
So What Happened?
What if I told you I’m still not entirely sure. Here’s another clip while you experience, with me, how frustrating having to say that truly is.
Here’s what we do know.
In the 60-day aftermath of the experiment, our shutdown of Facebook advertising had absolutely no immediate negative effect on sales. We saw organic traffic take a very slight dip (about 9%) but it has since more than recovered. We did see a significant drop in traffic from our remarketing ads and a subsequent, albeit unattributable, nearly 30% dip in Direct/None traffic.
Here’s what we don’t know.
While we’re relatively convinced that overall the leads coming from Facebook were of little value, I’m not entirely sure what the effect has been on branded and paid organic, direct traffic, or even paid search traffic. So far the data we do have is too jumpy to draw any significant conclusions, but it’s possible that the lack of exposure in Facebook is dragging the overall growth of our brand awareness, and reducing the number of our highest activating leads in a trickle down effect. That being said, Facebook remains off and we’ll continue to watch the numbers.
Hint: Stay tuned for Episode 2 of this Facebook saga, “The Aftermath”
It’s pretty simple. While we’re going to continue to trim and fine-tune our paid program, focusing on optimizing our sales process (essentially the very bottom of the funnel) will not only help us achieve the initial growth acceleration we’re looking for, but it will better prepare us to test and scale existing and alternative lead generation channels as our confidence grows in our ability to consistently convert trial users into customers.
With only 100 days on the job, my first major test probably generated more questions than it did answers, but it is saving us tens of thousands of dollars a month—funds I look forward to dedicating to future tests and increased investment in the channels/tactics that we find to be scalable.
And Finally, What Should You Take Away?
So, what’s the lesson in all this you ask? It’s a fair question. Were I you, I might take away the following…a few tidbits that might help you build and grow your digital strategy.
- Trust your gut, trust the test. Sometimes all the data in the world won’t give you a ready made answer. Don’t worry about it, test it.
- Embrace the rabbit hole. Buried in your data, even if you don’t have as much as we do, are incredible insights into the behavior of your audience. Take the blue pill and see just how far you get.
- Be patient and test one thing at a time. Keep your tests clean and isolate the variables you have the ability to control. No matter how well-defined you think your test is there are always going to be variables muddying the results. Do yourself a favor and do your best to keep it simple.
Writer's note: No digital marketers were hurt during the execution of this test or the writing of the post and I reserve the right to get smarter as likely half of what I assert will turn out to be incorrect—marketing is pretty funny that way.
Lastly, I love feedback. Have questions, suggestions, ideas based on what you’ve read? Share them with me!
P.S.: Hate your current CRM? Still working off spreadsheets? Register for our "Intro to Nutshell" live demo and see why sales teams love us!