A/B Test MDE Calculator

A lightweight interactive application for planning A/B tests by calculating the minimum detectable effect (MDE).
Shiny
R
Author

Aleksei Prishchepo

Published

August 7, 2025

Project Overview

The A/B Test MDE Calculator is a small decision-support tool designed to help analysts and product teams plan statistically sound A/B experiments. It focuses on one critical question in experimentation: what is the smallest effect size this test can reliably detect?

By turning power analysis into an interactive workflow, the application makes experiment feasibility and trade-offs explicit before a test is launched.

NoteRole

Product / Experimentation Analyst

NoteTools

R, Shiny

NoteDomain

Product analytics, experimentation, A/B testing

Key Features & Components

Read more about minimum detectable effect here

Minimum detectable effect calculation

Computes the smallest effect size detectable in an A/B test given sample size, variance, significance level, and statistical power.

Experiment feasibility assessment

Helps determine whether a planned experiment is underpowered or realistically capable of detecting meaningful changes.

Sensitivity analysis of design parameters

Shows how changes in traffic, variance, or power requirements affect detectability, supporting informed design decisions.

Live Demo

The application can be accessed online:

Implementation

  • Implemented in R using Shiny, translating standard frequentist power analysis formulas into an interactive application.
  • Designed with a minimal interface focused on fast iteration and clarity rather than visual complexity.

Find the source code in the project repository

Outcomes & Impact

Reduces the risk of running underpowered A/B tests that cannot detect meaningful effects.

Helps align stakeholder expectations with statistical reality before experiments go live.

Demonstrates how small analytical tools can meaningfully improve experimentation workflows.

Skills Demonstrated

A/B testing • Experimental design • Power analysis • Minimum detectable effect • Applied statistics • Shiny app development

Apply This to Your Business

If your organization runs A/B tests or experiments, feel free to reach out via contact page to discuss how similar tools can be developed to enhance your experimentation processes.

See Also

Back to top