sample project

Uplift modelling

June 03, 2016

We're (again) a major telecom operator. Just like pretty much any company in the world, we are concerned with keeping our customers happy, so they won't leave us. In other words, we want to reduce churn. To do this, we set up a task force of data analysts and people from our business teams who came up with several business goals to reduce churn.

So in a way described in the "Predicting churn project", we created a model to predict churn. In conjonction with the marketing team, we decided on a few actions to take for likely churners. In order to measure the usefulness of these actions, we set up an A/B test. A random sample of around half of the customers were targeted. the result of this A/B test is what we analyse in this project.

Before starting, we recommend you have a look at the following link to be familiar with uplift modelling theory.

Business goal

  • Target clients with more effective advertising based on their usage profiles
  • Retrieve customers with very high likeliness of churn so we could get in touch and offer them special deals before they even thought of leaving

How did we do this ?

  • The simplest approach consist in targeting likely churners. But likely churners may not be the people the more likely to react positively to our treatment. In our case we suppose there are 4 main types of people :

    • lost cause : highly likely to churn and unlikely to react positively to any marketing action. We should not bother targeting these people.
    • persuadable : highly likely to churn but likely to react positively to any marketing action. This is our real target !
    • sure thing : unlikely to churn whatever happens. Targeting them is a waste of money.
    • sleeping dog : unlikely to churn unless targeted by a marketing action. These are the people that had forgotten they are paying a subscription or hate to be targeted by marketing actions. This segmentation is not known before the A/B test. It is indeed a hidden variable that impacts our outcome : churn or not.
  • Contrary to churn modelling, uplift is about predicting the gain of probability when the user is targeted by a marketing action. This enable us to target the customers most likely to respond well to our actions.

Thus, if we denote X the covariates, T the treatment set (targeted by marketing) and C the control set, uplift is about estimating :

                 U = P(churn=0| X, T) - P(churn=0| X, C)
  • There are three common ways to do so :
    • model separately the two probabilities and do the difference
    • use a change of target variable
    • use specific machine learning models (out of the scope of this project)

Explore this sample project

  • Flow

    Start by taking a look at the flow to see the different steps of data preparation and machine learning that are needed for this project. Notice that there are two approaches. One is to create two models, one for the control dataset and one for the target (treatment) dataset. We then predict the outcome of these two models resulting in the dataset test_two_models_scored. The second approach is to model a transformed outcome variable. We'll compare this modelling technique to the first one by looking at the outcome in test_scored_modified_outcome_model. 

    Explore !
  • Dashboard

    Take a look at the distribution of the customer churn and customer types with the interactive charts available in the dashboard. 

    Explore !
  • Control Model

    Let's have a look at the performances of the control models to predict churn. 

    Explore !
  • Target Modelling

    Let's have a look at the performances of the target models to predict churn. 

    Explore !
  • Second Model Approach

    Next, have a look at the second modelling approach. We first modified the outcome variable (Script part of the analysis) and then modelled it in the Machine learning part of the studio. 

    Explore !
  • Models Evaluation

    Finally, check out the uplift models performances and comparaison by going back to the dashboard to the shared jupyter notebook (at the bottom of the dashboard) 

    Explore !

Ready to enter Dataiku DSS ?

If you never used DSS, it might be worthy to familiarize yourself with DSS concepts in the first place.

Learn the concepts Enter DSS

This sample is already available in your DSS!

From your DSS home page, click on "Sample projects".

If your DSS server doesn't have Internet access, you can download this sample and import it manually (click on "Import project")

Don't have Dataiku DSS yet? Try for free now

From your DSS home page, click on "Sample projects".

If your DSS server doesn't have Internet access, you can download this sample and import it manually (click on "Import project")

Don't have Dataiku DSS yet? Try for free now

Can't access DSS from a mobile device

Sorry please try again from a desktop device (Chrome and Firefox support only).

Only Chrome and Firefox are supported

Sorry you seem to use another browser not supported by DSS, please try again from Chrome or Firefox.