We have 20 years of experience of big data environments within a variety of industries including Research, Banking, Insurance, and Telecommunications. We have especially worked with customer data: Marketing, Risk Management, customer segmentation and -profitability, and customer-driven product development.
We have seen data mining and other analytics projects fail; we have seen insights teams unable to deliver the insights needed to actually improve the business; we have seen marketing teams unable to use data effectively to guide and quantify their activities; we have seen business leaders who are sitting on piles of data but are effectively flying blind because they can not get from the data to the knowledge they need to inform their decisions.
Below we have listed five common pitfalls of analytics in a commercial environment, their warning signs, and what you can do differently.
You may also be interested in our Reboot Analytics service, and we
would love to hear your pitfalls in the comments.
1. Not starting with the business actions
There are two pitfalls here. The first is that analytics can take on a
life of its own and end up becoming its own purpose. For example,
getting another nth decimal place improvement on the lift curve of
your customer churn prediction model can easily become an
obsession. However, it may not add much to the company’s profitability
compared with better understanding the actions you can take that would
change each customer’s propensity to leave and understanding what that
action would do to your profits. (See also our longer article on
Commercial Churn Modelling.)
Another example could be the search we sometimes see for “the one
true” customer segmentation. We are not against a good and useful
segmentation (see also our
5 step process for customer base segmentation), but where it
goes wrong is when it becomes the only segmentation model and when all
business has to wait for it to be completed, which often take a long
time. But customers will not wait and neither will most business
opportunities. Instead of strategic look at tactical segmentation
models that address specific business opportunities, and do so
quickly. There is not a single truth; only actionable insight.
The second pitfall is more subtle. Data mining with the objective of
‘finding something interesting’ is properly termed data
exploration. It can occasionally be productive, but rarely so. A clear
warning sign that the team has lost sight of the business is that they
spend too much time exploring and not enough predicting.
It is a more subtle pitfall because you do actually want the data to
help inspire your actions. But it has to be done quite systematically
to make sure the process does not run away to take on a life of its
own. You usually want a set of regular reports that are commercially
meaningful and look at deviations from norms and trends. We have
introduced our Insights Driven Campaign Creation process to
our clients with big success. It is a process for systematically
turning your data into insights and turning those insights into
campaign ideas that generate real and substantial money. It takes the
information from standard reports on customer Inflow, Base, Retention,
and Outflow (IBRO), turns that into insights about the behavior of the
customer base, and actions those insights through the whole marketing
campaign process from concept to cash.
What to do?
So what should you do? Start with the actions in mind. What is it you
can do that might make a difference? Where are your opportunities and
threats? This is a business question, not a data question.
If you are worried about customers leaving you, you may decide that
one thing you can do is to try to pay them to stay through loyalty
programmes (think airlines) or simply discounts. If that is the only
action you consider, then the questions of the data may be
What is the relationship between discount and change in loyalty?
What is the expected profit from the customer given a discount level?
In other words, if we give customer A a discount of $X, how much
longer will he stay and how much more will he buy, and is that a
profitable use of our capital? And what is this relationship for
customer each of our customers?
Those are reasonable questions to try to answer with data about your
customers. And they are quite different from the usual question of
”who will leave us?” They are questions about the effect of possible
actions which what they should be.
2. Not considering all the actions
Another warning sign for your analytics project is that you are only considering the actions and activities that you have always done.
In the previous example, bribing your customers to stay may not be the only (or best) possible action you could take. Consider a mobile telephone operator that may hypothesise that customers leave them for four primary reasons:
Better call tariff elsewhere
Better handsets available elsewhere
Lack of coverage / signal (e.g. at home)
You could (and should) break it down further. For example, a better call tariff from a competitor may be because the customer’s friends are all on the other network in which case the possible action may be short term to offer a ‘friends and family’ type discount and medium term to entice the friends to your network, perhaps though a ‘refer a friend’ scheme. Or something else: these are just examples.
The point being that you create a (reasonably) long list of possible actions and you then investigate each one of them for profitability. The actions should be limited and specific: if possible do not offer general discounts but tailor the offer to the customer’s need.
- Better call tariff elsewhere
- Create ‘friends and family’ plan
- Create ‘refer a friend’ plan
- Create international calling plan
- Promote soft benefits, e.g. exclusive access to events
- Better handsets available elsewhere
- Discount handsets in return for longer commitment (18, 24, … months)
- Explain benefits of handset range (Blackberry vs. iPhone vs. Android)
- Lack of coverage / signal (e.g. at home)
- Sell femtocells that provide local coverage using broadband
- Invest in infrastructure
- Other reasons
These are specific actions that recognise that customers are individuals and have individual needs, desires, and values. They are also actions that you can model and analyse from your data.
3. Not experimenting
You do not have all the answers. You do not understand everything about your customers. You need to experiment, experiment frequently, and not be afraid of (controlled) failure.
In Marketing there is a school of thought that periodically brings all the Big strategic managers into a Big room to think Big thoughts and come up with the one (or two or three) Big ideas that will save your business next year and really make a Big difference to your results. Because we know better.
Except we don’t. By and large we do not know better. Vision and strategy matters and you do need a sense of purpose and direction and a way of determining what you do not do. Ideas matter, and we are all for brainstorming and setting direction, but to think that you have all the answers is naive and to only focus on the few ‘big’ ideas is reckless.
Instead experiment. As long as you can control your execution and as long as you are rigorous in measuring (the right) results, experimentation is the way to learn. We built a whole civilization based on the scientific method and yet it is often ignored in a commercial setting by leaders who ‘know best’.
Fail early. Fail often. Enable yourself to execute many, frequent tests. Small, cheap campaigns to a limited number of people with careful data collection will get you a long way. You will be surprised by what you find.
4. Too much classification, not enough prediction
Ask your analytics team how much effort they spend on classification versus building prediction models. For most companies most of the time, the answer should be that almost all the effort is on the latter.
Are you drowning in the effort of producing ‘the one’ customer segmentation model? One company spent years and millions creating a single, global customer segmentation that was to be the basis for all Marketing and Product Development efforts. We ran the Marketing insights function for one of their countries and know that the only time we ever referred to this model was when writing reports to head office. It had no value to us in our work.
We see this pattern again and again. We are not against segmentation. On the contrary, we recommend that you have many segmentation models. Say, one for each /specific/ business problem or opportunity that you are facing at a specific point in time. Again: start with the actions and you will not get lost in the data.
We have written more about this topic at 5 step process for customer
What to do
A useful segmentation should take days to create; a perfect one eternity. Focus your classification effort on specific opportunities, not general questions.
An analytics team that is spending too much time classifying is often
one that has lost touch with the challenges that the business is
actually facing. Instead the team is fishing the data to come up with
“insights” to justify their existence (which is perfectly
natural). Reconnect the business and the analytics. Have regular
workshops considering the key challenges this week, month, or quarter,
and what the data may be able to say that can help you understand and
address them. Ideally they are interactive workshops with someone
having live access to the data and a suitable query and visualisation
tool so the questions can be answered and the ideas investigated then
5. The ‘always more data’ and ‘data is IT’ traps
We need more data (or better data or cleaner data). Heard that one before? We have yet to meet a commercial organization that made everything they could with the data that was already to hand or which could easily be obtained (e.g. by talking to a few or yours or your competitors’ customers or ex-customers).
It doesn’t mean that data is not important, of course. But that if you are focused on what it is you want to do as a business then you should be able to be pragmatic and creative about getting data quickly.
Quickly being the key. We are (or should be) trying to exploit an opportunity or counter a threat in the market, so the last thing we want is a 3+ year IT project to create the perfect single view of the customer. Again, long IT projects may be in order, but they should provide infrastructure that enables the business to change quickly and not be the change itself.
Data is a business project, not an IT project. Data is there for a reason, it serves a purpose. Only the commercial business can define that and prioritise what is important. IT plays a vital role in enabling the business to do this, but don’t leave data to IT.
You need to make trade-offs in getting the data. Perfection is not
required, commercial results are. If you leave it to a non-commercial
function (IT) to provision data, then they can only attempt to deliver
perfection, and that is doomed to fail. You want ‘good enough’ with
the flexibility to change quickly for most things. If you are trying
to market to your existing customers, then you probably do not need
data to more than 10% accuracy on most dimensions. Except for the one
or two dimension where you do need better data, of course. But it is
not fair or productive to leave that to IT.
What to do
Do not let lack of data be an excuse for doing nothing. You can do more with what you have got and you can easily and inexpensively supplement it with useful, if imperfect, additional information that allows the business to move forward.
Do not let your IT function own the data. The business owns the data and only the business can prioritise data.
Do work with IT to build the data infrastructure you need. It needs to be flexible and pragmatic. It needs to be accessible to the business and the business needs to be able to manage it. That is a journey; a programme of several focused projects each of which delivers a valuable business capability will take you there, but needs strong and experienced leadership.
Do work with all customer-facing functions within the business to
collect valuable data on your customers and their behaviours. Having
the right data and having the ability to use that to guide and inspire
your business decisions is one of the biggest sustainable competitive
advantages you can build.