Data gaps: how to identify and fill them
A data gap is the distance between the information you currently have and the information you actually need to make a confident decision. In practice it shows up in three forms: data you never collected, a mismatch in how much importance management places on a metric, and data you collected but misread. Identifying which gap you are dealing with is the hard part; once you know, filling it is usually straightforward with the right method or tool.
Good data lets management make better decisions, spot trends and opportunities, build lower risk action plans, and keep staff focused on the work that matters. Gathering data and acting on it should be an ongoing process, not a one off project. Below we explain the three types of data gaps and how to close each one.
What are data gaps?
A data gap is any point where your information is incomplete, misvalued, or misunderstood. e-tailize recognises three distinct types, and each calls for a different fix. Understanding which one you face keeps you from solving the wrong problem.
Missing data
Your organisation does not gather the full scope of available information. The data simply was never captured, so no amount of analysis can recover it until you start collecting it.
Data disparity
There is a mismatch between the importance that management allocates to a given metric or indicator. The data may exist, but it is over weighted or under weighted relative to its true impact on the business.
Misinterpretation of data
When the person analysing the data lacks the knowledge or skill to interpret or clean it, the result is the wrong conclusion and possibly a harmful decision. This is closely tied to data illiteracy inside an organisation, where the raw numbers are present but read incorrectly.
How to fill a missing data gap
Knowing what you do not know is the essential first step. Spotting the gap is the difficult part; filling it is comparatively easy, since software solutions and analytics tools usually do the heavy lifting. The right collection method depends on whether you need qualitative or quantitative data.
Here are reliable ways to collect the data you are missing:
- Surveys
- Online tracking on your website, for example Google Analytics or placing pixels
- Online marketing analytics
- Transactional data
- Customer data
- Software tools
How to close a data disparity gap
Data disparity is a problem of focus rather than collection. The fix is to agree on the few metrics that genuinely drive your business and to make sure management weights them correctly. Two ideas help here: choosing a single economic denominator and separating vanity metrics from actionable ones.
Focus on a single economic denominator
In the book Good to Great, Jim Collins examines what sets great companies apart from merely good ones. One thing the great companies share is that they focus their strategic effort on a single economic denominator. To find yours, ask: if you could pick only one ratio to systematically increase over time, which metric would have the greatest and most sustainable impact on your economic engine?
For example, Walgreens used to focus on profit per store, then switched to profit per customer visit. That single change reshaped its strategic decisions and led to much higher revenue and profit across the whole system in the years that followed.
Vanity metrics versus actionable metrics
The book Lean Analytics describes the difference between vanity metrics and actionable metrics, and understanding it helps you make better decisions. Most off the shelf analytics are useless for actually deciding anything. A vanity metric gives a feel good factor and a sense of general direction; it indicates the size of something and is typically a gross quantity. An actionable metric can be used for informed business decisions; it reflects individual behaviour and is usually a ratio or a piece of unit economics.
Examples of vanity metrics include trial users, page views, social media likes, email subscribers, leads in sales, marketing spend, and total customers acquired. Examples of actionable metrics include converting users, conversion rate, social media engagement and referrals, email opt in conversion rate, cohort analysis of the sales funnel, return on marketing investment, customer acquisition cost, and customer lifetime value.
The bottom line: focus on the right actionable metric for your economic engine, and communicate that choice clearly across your organisation. This is exactly why e-tailize concentrates on actionable metrics for marketplace management rather than surface level totals.
How to avoid misinterpreting data
Misinterpretation is an easy mistake, especially where data illiteracy exists in an organisation. Making a good data based decision requires two things: context with domain knowledge, and statistical competence. Get both right and you turn raw numbers into trustworthy conclusions.
Context and domain knowledge
A statistical model might suggest that cutting your price by twenty percent will lift revenue. In reality that outcome is not guaranteed. Understanding the domain, including the market and the competition, and understanding the context is critical to making the right call. The context here should be more data, both qualitative and quantitative, and it must be relevant and current.
Statistical competence
Statistical competence is built on knowing the related concepts and being able to think critically. A handful of concepts are essential to master before you trust your own reading of a data set.
Correlation is not causation
Two things moving together does not mean one causes the other. Take the statement that less sleep will cause you to perform worse at work. It may well be true, but to state a causal relationship as fact you need evidence from properly conducted research. For your business this means you should form a hypothesis by looking at likely causal relationships, then test it with an A/B test or research. Done correctly, you can use the results to decide, and you are now making decisions on a genuine data basis.
Some entrepreneurs prefer a gut feeling approach. Gut feeling is a great way to form a hypothesis, but you should still test it and decide on the results. That removes emotion and ego from the call, and over the long term it benefits the business.
Interpreting visualisations
There are many ways to display data visually, and you should know the common models, including the cohort analysis, which is easy to overlook. It is your job to choose the correct scale and alignment for the data; the wrong choice can mislead even when the underlying numbers are sound.
Recognising patterns
Your ability to recognise patterns depends on your experience with analytics, your critical thinking, and your knowledge of the domain. Look for outliers in the data set and understand why they are outliers; that often explains why the rest of the points are not, which makes the overall pattern easier to see. Understanding the maths and the structure behind a visualisation also helps you predict and recognise patterns.
Cleaning data
Always clean and filter your data set so it does not skew your analysis. Imagine you run a webshop and frequently order products yourself. If you do not filter out your own purchases, your data set no longer matches your real customer base, and you will decide on skewed numbers.
That is just one example of poor cleaning. Watch for these as well:
- Duplicates
- Errors
- Unwanted outliers, meaning a one off disproportionate value that offsets the rest of the set
- Missing data
Clean data leads to better decisions because it is more valid, accurate, complete, consistent, and uniform.
Filling the marketplace data gap
For marketplaces, the data gap is often less about collection and more about turning activity into metrics you can act on. e-tailize is a software solution for managing your marketplaces, and alongside integration it includes data analytics. We surface actionable metrics to analyse competition, market, products, ads, and more, and we provide documentation and tutorials so you get the most out of the visualisations.
Frequently asked questions
- What is a data gap?
- A data gap is the distance between the information you have and the information you need to make a sound decision. It usually shows up in one of three ways: data you never collected, a mismatch between the metrics management values, or data you collected but interpreted incorrectly.
- What are the main types of data gaps?
- There are three common types. Missing data means your organisation never gathered the full scope of available information. Data disparity is a mismatch in how much importance management places on a given metric or indicator. Misinterpretation happens when the person analysing the data lacks the knowledge or skill to clean and read it correctly, which leads to wrong conclusions.
- How do I fill a missing data gap?
- First, identify what you do not know, which is the hard part. Then collect it using methods that fit whether you need qualitative or quantitative input: surveys, online tracking such as Google Analytics or pixels, online marketing analytics, transactional data, customer data, and dedicated software tools.
- What is the difference between a vanity metric and an actionable metric?
- A vanity metric, such as page views, total customers acquired, or social media likes, gives a feel good factor and a sense of size but does not guide a specific decision. An actionable metric, such as conversion rate, customer acquisition cost, or customer lifetime value, reflects individual behaviour through ratios and unit economics, so you can act on it.
- Why does correlation not mean causation when reading data?
- Two metrics moving together does not prove that one causes the other. To claim a causal relationship you need evidence from properly conducted research. The practical approach is to form a hypothesis from likely causal relationships, test it with an A/B test or research, and decide based on the results rather than on the raw correlation.