Reducing Churn and Re-Engaging Customers with Data Science
Video with Thomas Schleicher, Ph.D., VP of Measurement & Data Science at National Consumer Panel
To say that the COVID-19 pandemic changed consumer shopping behaviors would be an understatement. The way people shop, where they shop, and how often they shop shifted radically from pre-pandemic to the early days of pandemic-imposed restrictions. Today, ongoing supply chain disruption, labor market shortages, and extreme inflation continue to impact consumer behavior.
Imagine the disruption this created for companies who rely on high quality market research. National Consumer Panel (NCP), a joint venture between NielsenIQ and IRI, felt the brunt of the pressure to continue delivering insights to their clients. NCP collects and processes a wealth of data from their select panel of shoppers, a representative “mini-USA,” to measure consumer attitudes and behavior. The data NCP generates is essential for their manufacturing and retail clients—it can be used to analyze market penetration, customer loyalty, and product propensity.
NCP’s #1 job is to understand how people shop. At the height of the pandemic, many dedicated shoppers reached out directly to NCP for guidance—how were they supposed to continue providing information about their weekly grocery runs when their entire lives were on hold?
NCPDisrupted Consumers + Disparate Data = No Easy Solution
When Thomas Schleicher joined NCP in 2011 as the Senior Director of the Measurement and Data Science team, his background in experimental social psychology and marketing analytics afforded him unique insight into the “why” behind consumer behavior.
In 2019, Thomas was promoted to VP of Measurement and Data Science, taking on more responsibilities related to data science leadership. Shortly after the pandemic hit, his role became more pivotal. How could he and his team leverage data to engage and retain panelists amidst a wide range of pandemic-induced economic pressures?
Thomas had some tools to get him started. NCP, like many other companies, has a wealth of rich, first-hand data. The issue? It comes from multiple, disparate sources. The data is organized by function, but, as business priorities evolve to keep up with (and ideally ahead of) marketplace developments, its quantity and complexity only grows.
What should he do with all this data? How could he leverage techniques like data mining and data science to discern patterns and optimize panelist targeting?
Data Science to the Rescue: Solving Problems Smarter & Faster
At this point, the only real certainty wasuncertainty. Thomas’s team’s biggest concern was to quickly ensure the number of reliable panelists remained stable so that NCP’s clients could create effective response plans, faster. He decided to partner withRapidMinerto use data science to help solve these problems more efficiently.
Develop More Tailored Incentive Programs
Knowing who your best customers are is the first step to retaining them. The best panelists transmit data at a steady rate, are responsive, and arenot容易找到,所以让他们订婚是数控之一P’s top priorities.
However, if a good panelist tapered off, Thomas’s team used advanced analytics to determine what factors led to their participation stalling. They analyzed a combination of behavioral, demographic, and other variables to determine what, exactly, influenced participation level.
With that information, they tested control incentive types and developed more tailored incentive programs for NCP’s top households to prevent them from tapering off as well.
Handle a Greater Volume of Data with More Agility
Remember that visual of the room full of filing cabinets? Thomas’s team used RapidMiner to save time going back and forth between file cabinets on opposite ends of the room, as well as doing all the heavy lifting analytically.
Using data prep, Thomas’s team better organized the relevant information they had, enabling them to handle a greater volume of data with more agility and sort through lots of data in an expeditious fashion.
The best part of these systems is that they’re sustainable. Thomas and his team can maintain their high degree of data agility today and in the future, especially compared to pre-pandemic baselines.
Analyze Textual Feedback
Using text mining, Thomas and his team converted raw text into numerical value, generating insights to help engage and retain panelists.
RapidMiner’s text analytics capabilities enabled Thomas and his team to turn this mountain of raw data into actionable insights. His team could track and forecast “new normal” trends as well as model and predict panelist churn. They could minimize efforts on time-consuming tasks like ETL, data prep, and data wrangling, as the RapidMiner platform remediated common issues like misspelled words and missing values to ensure the data was correctly categorized regardless.
This allowed NCP to generate clearly defined segments and create targeted engagement communications, further cementing and personalizing NCP’s relationship with their panelists.
Fostering Panelist Relationships, Crushing Business Goals & Other Measurable Impacts
Leveraging data science for text mining provided significant operational benefits for Thomas and his team at NCP.
One of the most obvious areas of impact was saved employee hours. “We definitely realized an efficiency in terms of work hours put behind the effort, and we were able to get more comprehensive results with the use of RapidMiner as opposed to a more manual approach,” Thomas shared.
Here’s a few other ways data science helped save the day:
Improved Panelist Participation
Due to an overwhelming number of identified responses related to COVID, NCP decided to introduce a “Did Not Shop,” or “DNS” code to promote better panelist participation. Even if panelists didn’t make it out to the stores, they were still able to engage with NCP.
As a result, NCP saw a 50% improvement in panelist transmission in the weeks following the shutdown.
Boosted Target KPIs
When the pandemic first hit, its impact on consumers caused dramatic swings in panelist data, deviating sharply from reliable KPI trends reported in years past.
By combining data from panelist transmissions, COVID survey responses, and verbatim panelist feedback (text data), NCP created targeted communications and incentives that really worked. These modifications helped reach impacted panel segments and led to top transmitting panelists’ KPIs returning to their pre-COVID trajectory within 1-2 months.
Optimized Collaboration Between Business and Data Experts
Text mining and data analytics don’t stop with data science experts. On the contrary—bringing these insights to cross-functional subject matter experts helped further optimize processes at NCP, contributing to improved KPIs and overall business decisions.
What Part Does Data Science Play in NCP’s Future?
While COVID-19 was the match that lit the fuse, many of these pandemic-induced changes are here to stay—and businesses need to figure out a way to adapt and keep their finger on the pulse of consumer behavior.
Thomas would be the first to acknowledge the rich history analytics had in driving success at NCP long before his arrival. With RapidMiner’s help, Thomas plans to enable NCP to get ahead of potential problems by exploring data patterns in new and innovative ways, unlocking opportunities to generate more value-added insights within the organization, and more importantly, for their partners and clients.
Considering working with RapidMiner? Watch the full video below to see what challenges our platform has helped the data science team at NCP overcome, and how working with RapidMiner has improved their team’s data science knowledge and skills.
00:04Hi, I’m Tom Schleicher, the VP of Measurement and Data Science at National Consumer Panel. I’d like to share with you my experience working with RapidMiner during my time at NCP. Before doing so, it will be helpful to share some background about NCP. NCP is a joint venture of IRI and NielsenIQ, two leading companies in consumer panel market research. Our longitudinal consumer panel data serves a multitude of clients, including retailers and manufacturers across numerous industries. NCP runs the operations needed to maintain this longitudinal consumer panel. It is geographically representative of the United States, and it is balanced on several census-based demographic characteristics. Our panelists, actual consumers across the US, send us detailed data regarding their shopping habits. They also respond to a variety of surveys regarding their purchase behavior. By continuing to meet KPI targets, NCP ensures high-quality data for our JV partners and their many clients. The measurement and data science team that I lead at NCP is responsible for a number of analytical tasks that help to ensure we meet or exceed our KPIs.
01:22To give a little bit of an overview of the analytics environment at NCP, it’s worth sharing that there’s quite a range from visualization to predictive models. Very important to our clients is our KPI reporting. That includes dashboard visualizations as well as benchmark indexing. And then there’s a host of separate but interrelated analytics that we perform on a regular basis, which includes panelist churn modeling, forecasting the need for new panelist recruitment, test and control comparisons that we might evaluate different promotions, sample selection that could be used to make sure we have balanced comparisons for surveys that our clients may be wanting, and text analytics. While my team’s focus is on the data analytics, NCP is fundamentally about data operations and client service. Data operations include the collection and cleansing of data and preparing it for distribution. It also includes multiple sources and multiple storage systems to house all of that data and, eventually, reporting that data back to our partners. Not only is that data feeding the deliverables to our clients, but it’s also important in terms of the updates that we provide to our partners. They want to make sure that we are adhering to certain KPIs and standards of data quality.
02:54大会党是面板操作常见的相关性business problems, I think, will be familiar for many of you. Our panelists are sort of like subscribing customers as such. As mentioned already, we do experience churn and have goals for retention of our best subscribers. We also go through CRM and lifecycle management topics. Panelists are also our data providers. As such, they are the source of our raw material. And we have issues of supply and demand for those panelists, and that also relates to issues of quality control as well as cost management. Finally, our raw data is processed into deliverables, our product. And that’s what provides value to our client partners and to their clients. It underscores the importance of the KPIs that we manage, too. And the variety of analytics that are in our toolkit helps to ensure smooth operations and the best quality data. With that as background, I’d like to answer a few questions to share with you some of my experience with working with RapidMiner in the time I have been at NCP.
04:11First, assess your current and future client needs. Understanding, especially, external client needs are of the highest importance, at least for us. But I think it’s also important to consider internal experts and clients. Subject matter experts that are a cross function can help you to understand how each department contributes to the success of your business. And if you’re not aware of that, you may benefit from having that kind of contact. It helps you to ensure that you have the right kind of data, and it also allows you to capitalize on synergies that may not be obvious if operating in silos. You may discover allies who may also help you advocate for any needed investment. Second, I would say focus your attention on solving a small but meaningful problem, and this will help you to keep your initiative manageable as well as to deliver the results quickly. It’ll help you to demonstrate value to senior management on a smaller investment first in order to make it easier later on when you want to get buying for a larger investment. You’ve demonstrated that you’ve added value on the smaller version. Third, I would assess your analytic bench strength. As needed, invest in training. It’s probably more important the more junior your team is. But even at more senior levels, it is still important to keep people fresh and up to date in this evolving field. And lastly, you could even consider consulting as needed.
05:44Well, I would first say evaluate not only the software, but the support. For me and NCP, RapidMiner was one of a number of possible options that we evaluated, and their support really was what set them apart from a number of other tools that had pretty good functionality as well. Especially early on when we were first investing in them, they invested time and expertise with us to help us achieve our analytic goals out of the gates. They were less concerned with upselling than providing hands-on consulting and shared expertise. And over the years, although some personnel have changed, the organization’s emphasis on customer satisfaction has remained. Another piece of advice I would extend is to take inventory of what your future analytical needs may be and not just your current needs. With the proliferation of data and analytical tools and solutions that are available, it is increasingly important to anticipate those future needs. Consider how well they may work with your current and planned IT infrastructure, and also consider how well it may work with any ETL and visualization tools that you may have.
06:58在大会党,我认为许多组织把gether several well-maintained but disparate data sources has been a significant challenge. My team has worked closely with our ETL team to build the right data warehousing business intelligence system and to pull all the data together. With the right data in the right place, NCP has leveraged RapidMiner’s analytics capabilities to address a number of common business questions, including predictive modeling, targeting and segmentation, churn reduction, and text analytics. Finally, beyond NCP, marketplace changes and data technology expansion continue to present challenges that actually present opportunities. And at NCP, we will leverage RapidMiner as an important tool to mine those opportunities for value.
07:51Well, RapidMiner allows us to handle large amounts of data and efficiently leverage a wide range of machine learning algorithms. For example, predictive modeling identifies characteristics of panelists most likely to perform well. That, in turn, has helped us to recruit panels that are much more likely to succeed and save a lot of investment in the process. Text analytics have helped us to turn panelist feedback into insights and actionable decisions. Specifically, combining what our panelists told us with some of our quantitative data has led to more effective targeted engagement efforts with our panelists, our customers. Data science knowledge then helps to translate into value for our business and coming up with a number of solutions including improved marketing effectiveness, reduced churn, improved cost efficiencies, discovered or uncovered new insights through cross functional synergies. And also it can help you to confirm what you know or, more importantly, disprove what you thought you knew as in the case when you might have been knowing something in the past, but it’s no longer true.
09:04Include predictive analytics as a reflexive part of your business process. Tap into stakeholders’ business acumen to inform your data inputs as well as to ensure that you have meaningful outputs that are generated. This also fosters cross functional buy-in and cultivates shared ownership. It helps to focus attention beyond any one department’s needs as well as driving better business decisions for the whole organization. Recognize that having a robust, advanced analytics platform is critical to efficiently address evolving business needs particularly during uncertain times. The marketplace and industries relying on data is increasingly competitive, and data analytics solutions continue to evolve as well as offer cutting edge solutions. If I were to sum it up, I would say having a tool like RapidMiner with the support they provide can help your organization not only keep up with the ongoing proliferation of data but, with the right focus, help to stay ahead of the curve. That’s all for now. I hope that you found my shared experiences partnering with RapidMiner at NCP informative. Thank you and have a great day.
