Growth Marketing

Advice on setting up your growth experimentation framework properly

81 calls booked about this topic

Facing a similar challenge?

  • Talk to someone who has been there before
  • Get unbiased insight from industry veterans
  • Work through the issue with a mentor by your side

The success of any growth experiment strongly depends on how well it is set up. Are you using a standardized and repeatable framework? Are your growth experiments prioritized with respect to other potential experiments that could also drive growth?

Here’s a simplified example of how to set up an experiment:

  • Experiment Title: A/B test if a chatbot is a good substitute to a form to drive more registrations
  • Step of the Pirate Funnel: Acquisition
  • Hypothesis: Our visitors are more willing to provide their personal details to an interactive chatbot rather than to a traditional form since our audience is used to interacting with conversational interfaces in their lives.
  • Implementation: We’ll build a chatbot using Chatfuel. This will ask our visitors the same questions as our existing signup form. We’ll use Google Optimize to split the traffic to the signup page as follows: 50% will see the form and 50% will see the chatbot. We’ll track the interaction in the page adding a heatmap (using Hotjar) and recording the sessions (also using Hotjar), and we’ll use Mixpanel to build a funnel showing which questions are harder for the visitors to answer and what is our conversion ratio.
  • Duration: 2 weeks (to get a data set of at least 1,000 people).
  • Win threshold: The experiment is a success (the hypothesis is validated) if the conversion ratio from unique page view to signup is 5% higher via the chatbot than via the form.

Once this process is internalized, you’ll start thinking about implementing features as an experiment. As a result you’ll start getting better at feature prioritization based on real data.

If you’re currently running growth experiments, speaking to a growth mentor can come in useful. You’ll be able to share with your mentor how you are structuring your experiments, what you’re measuring, and how you’ve split responsibilities across your team.

Getting a second perspective on your growth experimentation framework can help you head off potential mistakes down the road, increasing your long-term growth potential.

Jason Barbato

Former Global Growth Lead at
Built and scaled $40M, best-in-class growth hacking program at IBM. Head of Growth at early-stage startup. Freelance growth and marketing consultant for 6+ years.

“Jason was a rockstar. Asked lots of insightful questions, listened, and ultimately delivered 3 very actionable tips, that I am in the process of implementing. From start to beginning, great energy, I knew I was talking to a veteran in Growth Hacking space.”

Konrad Cichawa
Co-Creator Of VooDoo B2B Influence Project

“Just finished a great session with Jason. He was very tailored in applying his extensive experience/background to what I was looking for help on – hiring a remote marketing team. I could have easily chatted another hour! Super helpful, and an enjoyable experience with some clear takeaways. Thanks Jason.”

Martin Gorrie
Founder at AVA Digital Marketing

Michael Taylor

Co-Founder at
I founded and grew Ladder, a growth marketing agency to over 50 employees in three countries, helping companies like Monzo Bank, Time Out and as well as 200 or so other startups. I was an early adopter of Growth Hacking techniques and learned to code when Optimizely came to Europe so I could roll out my own A/B tests without needing IT. I've also met Sean Ellis and other prominent Growth Hackers, and worked for Noah Kagan at SumoMe.

“Had a great chat with Michael regarding app analytics and measurement! I loved his unconventional approach to certain metrics! Very much looking forward to our next call. :))”

Yasmine de Aranda
Chief Growth Officer at Huddol

“This session was really helpful. I needed to get more knowledge on tracking tools and more specifically on how to track new activity from affiliates. I liked Michael’s approach as it was a good mix between taking a step back looking at my need and giving context on each of the tool I could use, and more precise guidance on how to implement the recommended solution in Mixpanel. Thank you!”

Daniel Benamran
Founder at