“The test was significant — we’re rolling out.” In many companies, that sentence marks the end of an A/B test. Most of the time it comes too early. Statistical significance answers only one of many relevant questions: How large is the effect really? Does it hold for all user groups or only for some? Does it last over time? And above all: is the observed relationship causal, or just correlational coincidence?
I help companies design experiments that ask answerable questions — and evaluate results so that recommendations deliver what they promise.
What is causal analysis — and when do you need it?
Causal analysis is the systematic investigation of whether an initiative actually causes an effect — not merely whether it coincides in time with a change.
Experiments and causal analysis make sense when:
you want to know whether a product change, campaign or process adjustment really produced the observed effect;
existing A/B tests should be put to the methodological test;
no controlled experiment is possible, but causal conclusions are still needed (e.g. with historical data);
internal teams need support with test design or interpretation of results.
My approach
2. Test design and sample planning
I calculate the required sample size based on the Minimum Detectable Effect (MDE), the desired statistical power and the significance level. Design flaws cannot be corrected after the test — which is why this step is decisive.
3. Evaluation
Depending on the data and the question, different methods are used: classical hypothesis tests, Bayesian methods, difference-in-differences, propensity score matching or regression discontinuity. The method follows the problem — not the other way around.
4. Interpretation and recommendation
Results are not communicated as a binary “significant / not significant” but with effect size, confidence interval and a clear assessment of practical relevance. When the result does not allow a clear recommendation, I say so — with reasoning.
Methods and tools I use
I work with R (packages: MatchIt, DiD, rdd, bayesAB) and Python for causal analysis and experiment evaluation. For documentation and reproducibility I rely on structured analysis workflows that others can follow and review.
Who I work with
My clients are product managers, growth teams and marketing leads in startups and mid-sized companies in the EU who want to put their experimentation culture on a methodologically solid foundation — or who want to critically scrutinize the results of a test already run. I work project-based or hourly — remote or on-site in Berlin.
Reference projects and further reading
A/B Testing: Concepts and Techniques — a comprehensive methodological article on A/B testing, sample planning and common evaluation mistakes.
Minimum Detectable Effect (MDE) Calculation — how to calculate the MDE and why it must be fixed before the test.
Propensity Score Matching for Causal Analysis — causal analysis with observational data, when no experiment is possible.
Let’s discuss your project
Do you want to design a test, have existing results methodologically reviewed, or understand what an initiative really achieved?
Pricing & Packages
- Fixed scope, defined in advance
- 1–2 weeks duration
- 1 revision round
- Summary deliverable document
- Post-delivery support
- Iterative collaboration
- Complete project with iterations
- 3–6 weeks duration
- Unlimited revisions within scope
- Full documentation & handover
- 2 weeks post-delivery support (async)
- Ongoing monthly support
- Monthly, min. 3 months
- Continuous iterations
- Living documentation
- Priority access
- €90/h for additional work
- Monthly review call
Hourly rate for ad hoc requests: €90/h. All prices plus VAT.