Free A/B Testing Course: Step By Step Guide

Free A/B Testing Course: From Hypothesis to Decision

Learn how to design, launch and analyze web experiments with statistical rigor, no fluff, and with real examples.

8 lessons

The first lesson is open to everyone. To continue with the rest and save your progress, sign up for free with your Google account.

Many teams launch A/B tests and make decisions based on data that proves nothing: small samples, tests stopped halfway, poorly chosen metrics. This course teaches you how to do it right, step by step.

What you will learn

When an A/B test makes sense and when it doesn’t.
How to decide what to test using research and prioritization.
How to write hypotheses that can be tested.
How to calculate sample size and duration before you start.
How to interpret statistical significance without fooling yourself.
Which mistakes ruin an experiment and how to avoid them.

How it works

There are 8 short lessons. The first one is open; to continue with the rest and save your progress, you just need to sign up for free with your Google account. At the end of each lesson you can mark it as completed and track your progress on the progress bar.

Syllabus

1 What an A/B Test Is and When to Use It

What a controlled experiment is, what problem it solves, and in which situations you should not run an A/B test.

2 What to Test: Research and Prioritization

How to find real problems using quantitative and qualitative data, and how to rank your ideas with frameworks like ICE and PIE.

3 How to Write a Good Hypothesis

The structure of a testable hypothesis, good and bad examples, and how to link it to a metric.

4 Design the Experiment: Metrics, Variants and Traffic Split

How to choose a primary metric and guardrail metrics, define your variants and assign users without bias.

5 Sample Size and Duration: How Much Traffic You Need

What determines sample size, how to calculate it before launching, and why your test should run for full weeks.

6 Statistical Significance and p-Values, Without the Myths

What significance and the p-value really mean, and how the frequentist approach differs from the Bayesian one.

7 Common Mistakes That Invalidate a Test

The most frequent mistakes: looking at results too early, multiple comparisons, imbalanced traffic splits, and novelty effects.

8 Analyze, Decide, and Build a Culture of Experimentation

How to interpret winning, losing, and inconclusive results, document what you learned, and turn testing into a team habit.

What you will learn

How it works

Syllabus

Create your free account and save your progress