Market Data Analytics: Advanced Research Techniques
I recently had the chance to attend the American Marketing Association’s ‘Advanced Research Techniques‘ conference in San Diego, CA. The conference brought together approximately two-hundred data enthusiasts from a variety of positions in academia and industry. For the most part, I found the talks to be extremely relevant to my work on IBM’s Market Data Analytics team. Here are some of my key takeaways:
Although text mining might seem intimidating at first, there are well studied methods to provide quick and valuable analyses of text datasets. A basic approach, such as ‘bag-of-words‘ modeling, can be used to classify documents based on their content (i.e. customer feedback into the specific categories of issues) and identify the words that are most associated with those clusters.
Most visualizations created by Data Scientists and Analysts have traditionally been static with little interaction, but this is quickly changing. A tool called D3.js (Data Driven Documents) has recently become an industry standard for creating interactive visualizations on the web. Think about those visualizations on the New York Times. D3 code is easy to write, and the visualizations can be quickly shared via hyperlinks.
Although the factors that affect the purchasing behavior of an individual are often relatively well understood (i.e. price sensitivity, desire to continually purchase the same brand), real-word markets are often extremely difficult to understand. One approach to model scenarios like this is through agent-based models. These models start by defining the preferences of individual buyers, then use simulations to determine the aggregate effect on the entire system. For example, by knowing about the distribution of individual preferences for different types of car, we can run a simulation to determine the effect of a competitor price change on sales.
Other topics at the conference included probability models for estimating customer-lifetime-value, as well as various methods to decompose the effect of marketing campaigns on sales. Overall, I was pleasantly surprised by the practical nature of the talks, and look forward to applying these new techniques to my work.