ConversionXL, Chad Sanderson – Advanced experimentation analysis

Go way beyond what most optimization experts know
Online course:
Advanced experimentation analysis

By Chad Sanderson, Microsoft Experimentation Platform

8-class course

Some of the companies that train their teams at CXL Institute:
From zero to hero in self-serve experimentation

After you take this course, you won’t need to rely on what the testing tools say about results.

You’ll be able to go into the data yourself, understand it, query it, and perform advanced statistical techniques to get more value than any tool can really give you. That’s independence.
Get to the top of the food chain in terms of the best CRO practitioners in the world

Having the ability to dive deep into experimental statistics and analysis using data and code is a skill that very few people have. You will be one of those rare phenomena. This will put you many steps ahead of all those optimization experts out there.

In this course, Chad will peel back the curtain on the world of advanced experimental statistics, data science, and discuss deep experimental understanding. All using common sense language.

Learn what makes data analysis so important and why we should think very carefully about building our own metrics
Understand how to generate sample size (the right way)
Master principles of analysis and metric building
Discover how to build, launch, and analyze experiments offline
Use statistical techniques like regression and time series analysis for CRO
Go beyond traditional A/B testing, focusing on unconventional methods that maybe you haven’t heard of before
Generate powerful custom charts to communicate testing results
Start working with R (even – or especially if your most hated thing is jumping into code since you’re not a developer).

This course will teach you how to go beyond traditional A/B testing with advanced statistical techniques (no math background required), clear away numerous misconceptions about common and uncommon testing tropes, and open your eyes to the business-wide transformative strength of experimentation.
Watch the course overview video
Ideal for you if…

Get immediately download ConversionXL, Chad Sanderson – Advanced experimentation analysi
You are a conversion optimization specialist looking to dive deep into experimental statistics and analysis using data and code
You have a hand in analyzing, interpreting, or reporting the results of digital experiments
Your business or client fully believes in the power of experimentation

This course is probably not for you if…

You are an experienced data-scientist, researcher, or statistician with a deep knowledge of experimental design

You are a CMO or VP. This course is less strategy, and more hands-on functional knowledge.

You are content with the surface level testing results reported by your testing tool

Skills you should have before taking this course:

Intermediate A/B Testing experience. We won’t be covering how or why you should be setting up experiments or using a testing tool.
Foundational understanding of analytics best practices and reporting. We expect you to know how to choose and report on important metrics.
Working knowledge of Microsoft Excel or an equivalent. We will be working with data!

Meet your instructor, Chad Sanderson

Experimentation Program Manager @ Microsoft

Chad Sanderson is a Digital Optimization, Testing, and Personalization specialist focused on the strategy, design, implementation, and analysis of winning experiments.

Chad is a passionate believer in the power of data, combining deep actionable insights with UX and Content strategy to create better websites and mobile apps.

Chad has the ability to explain the power of optimization on any level, to the statistics-minded data scientist to the company’s most important stakeholders, and can create the necessary processes to imbue the company culture with an optimizer’s mindset.
Your full course curriculum:
Advanced experimentation analysis

Lesson 1
Introduction

In this intro lesson, Chad explores the ‘why’ of experimentation analysis. When you come to understand the fundamentals, you will learn how to set you and your team apart when launching and managing experiments. By beginning to understand the various approaches of experimental analysis you will discover just how deep the rabbit hole goes.

Topics covered:

Who am I?
What is this course?
What will you learn?
Why is experimentation analysis important?
Testing beyond your tool

Lesson 2
Principles of Analysis & Metric Building

In this lesson, Chad dives deep into what it means to build metrics and why that’s so fundamentally important to experimentation analysis. Analytics in short is looking to data for insight, which is critical in the context of experimentation. Begin to identify what you could be doing and learn how to work on metrics you actively create.

Topics covered:

Why do we need to prepare data?
Formatting data in excel
What is R Studio
Installing R Studio
Reading data
Practice/case-study

Lesson 3
Diving into Experimentation Analysis

Topics covered:

What is sample size really
The components of sample size
Binomial metrics explained
Sample ratio mismatch
Continuous metrics explained
Practice using R Studio
N2/N1 Ratio
Practice/case-study

Lesson 4
Deep Experimental Understanding

In this lesson, join Chad as he touches on the immense value of randomization and representational data and how to avoid alpha inflations when creating an honest analysis.

Topics covered:

Ttest/Ztest
X2 Test
Executing tests within R
Segmentation using dplyr
Adjusting p values
Confidence Intervals
Plotting using ggplot2

Lesson 5
I Have the Power (Analysis)

Topics covered:

Sample size
Size effect
Statistical calculators
Statistical power

Lesson 6
Beyond A/B testing – Part 1

Topics covered:

What can’t we experiment on using an A/B Testing tool?
Regression Discontinuity Designs and Why We Use them
Interrupted Time Series Analysis
Practice/case-study

Lesson 7
Beyond A/B testing – Part 2

In this lesson, Chad delivers practical advice and tips on experimentation for your organization at large. The lesson touches on areas of experimentation that may put your business at risk and ways you and your can team can design high impact experiments.

Topics covered:

Other forms of RDD
Practice/case-study

Lesson 8
Parting thoughts

Join Chad in the last lesson to cover all the extra details and quality controls needed for proper experimentation analysis.

Topics covered:

Control vs. Treatments
Setting up proper controls
Response Surface Methodology
Experimentation Brain Storming
Testing offline

Show off your new skills: Get a certificate of completion

Once you’ve completed the course, pass a test to get certified in Account based marketing. Add it to your resume, your LinkedIn profile or just get that well-earned raise you’ve been waiting for.
CXL Institute is an authorized LinkedIn education provider

You can add your education, certificates, badges – everything you learned and earned at CXL Institute into the Education section of your profile.
How to find CXL Institute on LinkedIn

Get this course and 50+ others when you sign up for CXL Institute

This course (and all others – full list here) are included with the CXL Institute subscription.

At CXL Institute, you can take online courses on growth, digital marketing, optimization, analytics, persuasion – all in a single subscription.

World’s leading practitioners teaching you their best stuff.
50+ online courses on all things data-driven marketing
New courses added every single month

Sign up for CXL Institute and get access to

50+ courses (full list here) on all aspects of data-driven marketing (new courses added every month)
4 Minidegrees that give you advanced level skills
Talk recordings by world class practitioners at CXL Live and Elite Camp

Got a team, multiple users? Get our Teams All-Access plan instead.
The CXL Institute guarantee:

7-day no questions asked money back period
You can immediately apply learnings and improve your results.

Need help convincing your boss?

---