Before You Build the Dashboard, Decide What It Has to Prove

Ask a stakeholder what they need from a product's data and there is a good chance they will say "a dashboard". It sounds concrete and it can be delivered in a sprint or two. It can also be a very efficient way of producing charts that change nothing.

The data work that has helped me most started with a specific operational question, and only then moved on to how to display the answer.

Measure the pain before you fix it

At my previous Government role, onboarding a government agency involved a partnerships team, product teams such as Proof of Identity and Profile Connect, and my squad. Roles were unclear, and it felt chaotic, but feeling chaotic is not an argument. The useful number was this: we were averaging 2.7 hours per incident on coordination, causing delays of one to two days and sometimes a week.

That turned a vague frustration into a problem with a size. I founded an onboarding squad as a buffer, with defined roles, a single ServiceNow process aligned to the ITSM maturity model, an onboarding checklist and published knowledge articles. I piloted the service as a product with Flood Grants and the Licensing Program.

I also set targets in advance: cut direct contact with product squads by 50% and improve resolution time by 30%. We achieved about 80% and 50%. Setting those targets first matters, because a number chosen after the work is finished is only a story.

Observability is a product decision

Later, the Product environment lacked visibility into system and product performance. That left the onboarding squad exposed to rising support queries, and it affected agency onboarding and the wider customer experience.

Nobody had asked for an observability dashboard. I engaged the relevant product and infrastructure teams to build one in Splunk, monitoring and alerting on Service Orchestrator utilisation and performance.

Visual indicators helped reduce support requests by 50%. They also saved at least a couple of business days in resolving an issue that would otherwise have stopped LicenceNSW teams from going live. That is what a useful dashboard looks like to me: someone acted differently because it existed.

Validate the data before planning the visual

More recently, on a government data product, I faced a long list of possible dashboard and analytics requirements. Building visualisations before validating the data and the baselines risked creating something attractive with no reliable evidence behind it.

So I reworked the backlog around operational problems such as cycle time, data quality, errors and manual intervention. I also separated outcomes from implementation choices, and asked engineering to validate data availability, dependencies and feasibility before sprint planning. That moved the conversation away from "build a dashboard" and towards whether the product was actually reducing delays and manual handling.

Five questions to ask first

1. What decision will this change? If nobody can name one, stop.
2. What is the baseline? Without it, you cannot show improvement.
3. What is the target, and when was it set? Set it before delivery.
4. Is the data available and trustworthy? Have engineers check before the sprint starts.
5. Who acts when the number moves? A metric without an owner is only decoration.

A note on Agile and data

Agile teams often say they are data-driven while mostly tracking delivery activity: velocity, tickets closed, story points. Those have a role in planning. They do not tell you whether anybody's life improved.

The harder and more valuable question is whether the thing you shipped changed the outcome you said it would. If your reporting can answer that, it will be worth far more than an impressive set of charts. If it cannot, you may simply have built a nicer way to look at activity.