Why Two Dashboards Disagree, and How to Fix the Data Behind Them
When reports disagree, check the metric definitions, source records, and update times. A shared definition and a small set of quality checks can make the difference understandable.
- Start with the business meaning
- Write a definition people can use
- Check quality from several angles
- Follow the records through the pipeline
- Show users what they need to interpret
- Resolve one disputed metric first
Start with the business meaning
Two dashboards can show different numbers without either containing a calculation error. They may describe different events, periods, or groups of records. Changing the chart will not resolve a disagreement about what the metric means.
In a fictional distributor, sales reports orders placed this month while finance reports invoices issued this month. An order placed on the final day may be invoiced the following week. Before comparing totals, ask both teams which business event they intend to measure.
Write a definition people can use
Document the metric's name, purpose, owner, calculation, source, date field, unit, and update schedule. Specify which records belong in it. For an order total, decide how cancellations, returns, incomplete orders, and taxes are treated.
Make the definition concrete enough that another analyst could reproduce the number. "Monthly sales" leaves room for interpretation. A description tied to a specific event and inclusion rule gives the dashboard reviewer something they can check. Record time zone choices where they affect reporting boundaries.
Check quality from several angles
The UK Government Data Quality Framework describes six useful dimensions: completeness, uniqueness, consistency, timeliness, validity, and accuracy. Each reveals a different kind of problem, so one percentage cannot establish that a dataset is suitable for every purpose.
For the fictional distributor, check whether all expected records arrived, whether order IDs repeat, and whether statuses agree across systems. Then check the latest available period, whether values follow permitted formats, and whether a sample matches the original records. A complete table can still contain inaccurate amounts.
Follow the records through the pipeline
Trace a small sample from the source system through imports, transformations, and the dashboard model. Look for the stage where records disappear, duplicate, or change meaning. Keep notes so the explanation can be reviewed by someone else.
Suppose a join matches one order to several shipment rows. Summing the order amount after that join could repeat its value. This is an illustrative failure to investigate, not a diagnosis of every mismatch. Other discrepancies may come from filters, delayed imports, or differing source definitions.
Show users what they need to interpret
Give the dashboard a visible refresh time and an accessible metric definition. If a known issue affects a period or business unit, describe its scope near the affected result. Avoid a vague warning that leaves readers unable to judge whether their decision is affected.
Assign an owner to each recurring quality check. Track the issue, impact, proposed action, and evidence of resolution. Where possible, address the cause in the source process so the same correction does not have to be repeated in every report.
Resolve one disputed metric first
Choose a number that already causes confusion. Bring its users and data owner together, agree on the definition, and compare a small set of records. Use what you learn to establish repeatable checks.
Success begins with an explanation people can verify. Once the definition and the path the data takes are clear, improving the model or presentation becomes a more focused piece of work.
Further reading
Primary references behind the technical guidance in this article.
Put the thinking into practice.
Explore our data analytics & bi services, or talk to us about the task you want to improve.
Explore the service
