Marketing Agility, Part 2: Analytics

In a previous post, I discussed the concept of marketing agility from the perspective  of data. Next I’ll turn my attention to analytics, and how they promote greater agility in marketing.

Reliability

Analytics agility "stack"

What we desire from any analysis, whether it is a simple report or a complex statistical model, is that it provide reliable information for making business decisions. This notion of reliability has two sides: on the one hand, we want to make the “right” decisions, ones that provide some benefit to the organization; and it is even more important to avoid bad decisions that cause harm to the business.

Notice that I did not use the word “accuracy” to describe this quality. While a simple report might be said to be accurate when it correctly restates data from a database, with any sort of model, what we get is an approximation – a description – of the underlying data. It’s why we talk about “fitting” a model to the data.

Even a good model will have conditions under which is inaccurate; as long as these conditions are recognized and are reasonably consistent, then we can rely on the model for helping us in making decisions.

The definition of what makes a model reliable merits a whole separate discussion, and quickly can lead to arguments of nearly religious fervor among analysts. I have found it more useful to employ more generalized measures of reliability.  For instance, lets’s say we have a model designed to predict the amount customers will spend. One approach might be to use the point estimates of such a model (e.g., Customer A is expected to spend $101; Customer B, $79, and so on).  Another approach could be to use the relative rank ordering: Customer A will spend approximately 25% more than Customer B.

In many cases, such a model will be more reliable when rank ordering than it is in making point estimates, and will be just as useful for making decisions, which in this case is that we can invest more in Customer A and expect a higher return on the same investment.

Stability

This example leads to a related notion: model stability. In most marketing environments, the conditions underlying a model change over time. Such changes arise from many sources: there are arbitrary changes to the data; population composition evolves over time; and new competitive, regulatory, or macro-economic events arise. What we need is a model that demonstrates resilience to such changes, so that it remains reasonably reliable for some time. Again, all things being equal, the predictive model of future value is likely to be more stable over time when rank ordering than it is in making specific estimates.

Here, too, approaches will vary. For example, it is often necessary to make sure that time series data covers a sufficient amount of time to capture the underlying processes or to guard against such factors as seasonality. Otherwise, a model might be considerably less accurate during some times of the year or at different stages of a customer’s lifecycle.

In some circumstances, particular techniques may be helpful in promoting stability. For example, sometimes useful attributes exhibit high cardinality: one company we helped several years ago managed local advertising in about 1,000 markets and in over 3,000 product and service categories. A typical approach would have been to cluster “similar” markets and categories together in order to reduce cardinality, an approach that might have produced a reasonably reliable model in the short term, but that would have been vulnerable to market- and sector-level changes that would have affected the reliability of the clusters.

Ultimately, we used a Hierarchical Bayesian model that preserved the information, allowed the model to be more resilient to changes in markets and categories, and even permitted us to handle new values (e.g. a new market that the company had entered) by assigning them the Bayesian average for the population. The result was not only a model that made reliable predictions now, but one that proved stable over time.

Agility

From an analytical perspective, agility arises when the decision-making process becomes more efficient, so that a company can react to changes in the market in ways that protect and increase competitive position.

Regrettably, I find that many organizations have developed a reporting culture, rather than an analytical culture. In other words, much decision-making relies more on reports of past and current activity, rather than on analysis that reveals underlying associations and longer-term trends.

Here’s an illustration: let’s say the September sales figures for our company show a 5% decline from August. In some companies, this would lead to immediate panic. Some managers might question the reliability of the report, and ask for the numbers to be reproduced, something that might take several days. If the results were confirmed, management would then embark on a search for the potential causes of the decline, discuss strategies to correct the problem, and then work to design corrective tactics. Implementing changes might take weeks or months longer, by which the situation may have deteriorated even more.

In a more analytical culture, such a decline would have been anticipated. For example, the company might have invested heavily in acquisition in the preceding year, have noted that this cohort of new customers was not repurchasing at the expected rate, and that the unusually large number of acquisitions in September of last year, would lead to a decline in renewals this September. Management would not only have anticipated the decline, but could have worked to mitigate its effects, perhaps by increasing retention efforts among newer customers or by replacing them proactively.

I recognize that this is a somewhat extreme example – only the most obtuse organization would have no inkling of trouble of this magnitude ahead. But on a smaller scale, such reporting tunnel vision is all too common. The expression “paralysis through analysis” should perhaps be changed to “paralysis through reporting,” since a proper approach to analysis has quite the opposite effect. Indeed, achieving marketing agility increasingly requires companies to build an analytical discipline rather than merely a reporting capability.

As company consider what analytics to invest in, I suggest that they focus first on activities that create greater agility on a business level: does forecasting potential price increases let one lock in rates, making one more competitive in the near future? Does better customer research lead to more rapid product design? Does a customer segmentation allow one to create marketing programs more flexibly?

From there, each new analytical initiative and output needs to be vetted for its reliability and stability. Moreover, these factors need to be monitored, so that the analytics continue to be useful to decision makers, and ideally, actually are improved over time.