In the last post of this blog series on being successful with digital analytics, I showed how to update your SDR as you implement new items. That is one way to boost your overall implementation scores with respect to the percentage of business requirements that can be addressed. The other way you can boost your scores is to ensure that the data points you are collecting are accurate. Even if you have the data points you need to address a business requirement, if the data is incorrect or untrustworthy, you still cannot fully answer the business question. Therefore, in this post, I will share some thoughts on how to focus on data quality.
Data quality in digital analytics is the elephant in the room that no one likes to talk about. But in my career, I have audited hundreds of digital analytics implementations and have yet to find one where the data was always trustworthy. The most common issues I encounter are:
- Metrics that have inconsistent numbers, wild swings in numbers (i.e. YoY) or suddenly stop collecting data
- Dimensions that are missing values for metrics that are logical counterparts (i.e. No Product ID with Add to Wishlist metric)
- Dimensions that are capturing unexpected values or values that are not adhering to a predefined naming convention
- Implementation mistakes that negatively impact data (i.e. Adobe customers not using Merchandising eVar when it should be used or using an sProp when an eVar should be used)
So why do so many organizations have data quality issues? I have seen several reasons. First, so much time is spent at organizations implementing digital analytics tools, that teams are usually exhausted and want to put the implementation behind them. The prospect of continuously checking and re-checking data is not an appealing one. It is like asking someone to run a 5k race each week right after they ran a marathon! Second, organizations love to do website or app redesigns and these major changes can wreak havoc on digital analytics data if your implementation hasn’t been set up in a robust manner (i.e. data layer approach vs. DOM scraping). Third, data quality is not generally a focus of executives. Executives tend to focus on the big things like getting the newly redesigned website live by June 1st more than they do about the details of data quality. I will share some strategies on how to change this in the next post.
So why is digital analytics data quality so important? Many years ago I called data quality the silent killer of digital analytics. This is because poor data quality slowly sucks the life out of digital analytics implementations. If your internal users log into your analytics tool and encounter data quality issues, it negatively impacts the overall perception of your team and your analytics program. When your business stakeholders utilize your data, you are, in effect, asking them to make business decisions that will impact their reputation (and possibly career) based upon your data. That is a big ask. If they start to sense that the data is not trustworthy, they may stop using your data and either revert to their “gut” or look for data elsewhere. If that trend becomes pervasive within the organization, your investments in digital analytics will all be for nothing. It will impact your budgets, your team headcount, but most importantly your team reputation and morale. As they say, a reputation takes a long time to build but only a few moments to lose.
I wish there were an easy way for me to tell you how to address data quality issues. Unfortunately, dealing with data quality simply requires time and resources. You can set up alerts within your analytics tool or standard deviation formulas in spreadsheets to proactively see if metrics might be experiencing issues. Validating dimensions and implementation issues is much harder. You can spot check dimension value/metric combinations or invest in 3rd party tools like Observepoint or Alarmduck. My goal of this post is not to show you exactly how to address data quality issues since every organization has different needs, but rather, to make you aware this data quality needs to be a key part of your analytics program.
Action Items
Your task for this post is to do a data quality audit on your implementation:
- Check all of your metrics and see if there are any that look suspicious. I suggest that you check a few data points each week instead of attempting to check all of them every few months. The latter has a tendency to get pushed off in my experience.
- Open your dimension reports with metrics that have logical pairings and see if the data looks ok and if the values are understandable.
- Consider making automated spreadsheets that can download the data you reviewed in the previous steps on a recurring basis or if that is too much work for you, check out some 3rd party tools that can help.
In the next post, I will discuss some strategies to get your executives to care more about data quality.
