“A theory has only the alternate of right or wrong. A model has a third possibility - it may be right but irrelevant”. - Manfred Eigen
Very true for predictive models as they use past consumer data to predict future consumer actions. But in today’s ‘COVID-ified’ world, consumer behavior and preferences itself are changing. Surge in online purchases, increased digital content consumption, adaptation to a work-from-home environment, and so on, are testimony to this behavior shift.
Given this shift, relying on pre-COVID19 predictive models could lead to inaccurate and irrelevant business decisions. The model outcomes could be theoretically right but contextually irrelevant. For instance – in a pre-COVID19 world, bulk purchasing juice could trigger product recommendations for party items – disposable glasses, tissues, etc. However, these recommendations may not be relevant in today’s COVID19 context. Here’s why…
- Model predictions are based on the power of a variable:
Since consumer behavior and preferences are changing, the power of a variable to predict may be altered. For example, bulk buying juice may previously have been a key variable to buying paper glasses. Not necessarily true anymore.
- Model predictions are based on distribution of population for a certain variable:
A change in consumer behavior would affect the distribution of the population for a said variable, and hence, for the model as a whole. Let’s say a model uses customer purchase to indicate purchase propensity for the next product. For this model, in the juice example, say we set a threshold score that offers promotional discounts to the top 20% of customers buying three gallons of juice. In the current context, as many people prepare for an extended lockdown, there may be a steep surge in the number of buyers who bulk purchase three gallons of juice. Therefore, many more people would score higher on the model. If the model is not refreshed, a lot more people than planned would be offered promotional discounts, making it less lucrative.
- Models are built based on a homogenous set:
Again, since consumer behaviors are changing, some customer segments may become redundant, while others will need to be refigured. For example, young, single people living on rent, buying two gallons of juice is not the same as a family buying two gallons. More importantly, young single people may not even be a relevant segment anymore as they could have moved in with their families.
The need to refresh predictive models in times of crisis is not a new phenomenon. When the world faces such major disruptions, old business models and predictive models need to be broken down and rebuilt. A milder version of this was seen back in the 2008 financial crisis. Credit worthiness models used by lenders to offer loans started breaking down as the real estate market collapsed. Deeper research and model monitoring showed that one of the key variables used in many such models was the presence and size of mortgages. This variable had lost all correlation to credit worthiness as leveraged home-owners now abandoned devalued homes.
The COVID-19 crisis is similar, only much graver. Unlike the 2008 economic crisis, COVID-19 is a crisis on more than one level – health, humanity and economy. Globally, countries are still grappling with ‘what next’. The future is unpredictable. Yet, businesses could benefit from predictive models. But only if the models are refreshed or should we say COVID-ified.
Stay tuned for tips on how to refresh predictive models.