Analytics Recipe Cookbook - Summaries

Identify True Lies of AB Test Results

  • 1.  Identify True Lies of AB Test Results

    Posted 08-09-2018 01:18 PM
    Learning from a failed A/B test is like solving a murder mystery. You need to treat everything as an evidence and preserve your findings to crack it. Wear that Sherlock's hat.

    A lot of time, effort and money goes into analyzing web data collected by an analytics tool and in identifying optimization ideas for any website. Relying on an evidence-based practice, we AB test these ideas using an AB split testing tool. To confidently conclude any of the experience as a winner, it becomes very important to derive complete information about the AB test results over and above the conversion goal results.

    Analysis Overview:

    The recipe discusses how to go about correctly identifying and evaluating all the success metrics concerned with the experiment to get a comprehensive understanding of how the variant panned out when compared to the original.

    Analysis Benefits:

    1. Get a wholesome understanding of the test results
    2. Reduce errors in judging the winner.
    3. Decide on the future course of action more confidently - whether it be declaring the variant as a complete winner, or in the case where the variant is not a complete winner, then in designing a better variant based on the learnings of the test
    DAA members, go here to view full recipe.

    Shamyaditya Deo
    Nabler (Corporate Account)
    Essential Series: Applying Digital Analytics