A friend of mine told me about some growth hacking consulting work she was doing for a company. She was brought in to teach them the basics of growth hacking a marketing message and provide them with a plan to get them started.

A few months after her engagement, she ran into the CEO at an event asked how things were going.

“Terrible. Nothing we try works. I’m thinking of scrapping all paid social media and buying some print.”

At that moment, she realized two things: First, the CEO was most likely an idiot. Second, something had gone terribly wrong with her plan. Knowing her reputation was on the line, she wanted to get to the bottom of the problem quickly.

She made a few calls and discovered her complex messaging vs. audience test plan got scrapped for a spray and pray campaign. The only thing, the company was doing right, was A/B testing some creative. The worst part. The people, she had taught, had NO idea they were doing it wrong.

She, of course, offered to set them straight for no fee. (I thought this to be way too generous, but she is a good person.)

The company had ZERO appetite for her advice.


“We’ve already tested! We’re not going to go back and start from scratch! That would be a colossal waste of time!”

Translation- It would bruise my ego to admit that we fucked this up and would rather ignore it than get it right.


After some reflection, my friend came up with two lessons learned:

First, she should try to avoid working with idiots, an occupational hazard for consultants.

Second, she had missed something very basic. She had forgotten to include a measurement in how well the marketers were executing their test. In technical terms. She failed to close the feedback loop.


Measuring the effectiveness of your testing matters!

Let me give you an example.

In your planning meeting for your upcoming marketing sprint, you come up with two ideas and Bob came up with three on how to test a new marketing message. You think

Bob is an idiot mouth breather who doesn’t wear a belt.

Your job is to run these tests. How much time are you going to put in on Bob’s ideas versus yours? How much time are you going to spend on the creative? How much time will you spend on perfecting the message on his ideas?

In my experience, marketers and growth hackers rarely take the time to measure their effectiveness in the test when that is often one of the biggest factors in the outcome.

In this example, what if Bob’s idea was a home run and you didn’t execute the test well?

You don’t KNOW anything and the only thing that matters… THE ONLY THING THAT MATTERS IS WHAT YOU CAN PROVE!


Lack of execution is a much bigger problem than most people fully understand. It is likely that whomever you report your findings to will assume that you executed the test well. Those people often make big decisions based on a poorly executed test.


Every one of your tactics needs one additional measurement. Answer the simple question-> On a scale of 1 – 10- How well did we execute that test? After you have answered that question, ask a some folks outside your building. This poll doesn’t have to be a formal focus group. Send your test to a group of fellow growth hackers… they will GLADLY give their candid opinion!


Regularly measure your marketing tests. Put on your big boy or big girl pants and seek real honest feedback to how well you execute each of your tactics.

Only then can you PROVE (or disprove) your thesis and PROOF is the only thing that matters.