The right data at the right time in the right hands supports better outcomes

To tackle entrenched disadvantage amongst children and families in Australia, we need to have consistent and timely information to know what is happening and where, but we don’t!

• SVA Quarterly
• Measurement and evaluation , Systems change

To create a fairer Australia, children and families facing disadvantage require high-quality services during the early years.

Unfortunately, those working to improve early childhood services often lack the timely data, resources and support needed to focus their efforts effectively.

The Productivity Commission’s final report¹ on Early Childhood Education and Care highlights that better, consistent data together with an investment in capabilities to use it are essential for stewarding a system to improve children’s outcomes. 

This article outlines some of the key problems with the current approach to data use and describes how Restacking the Odds (RSTO) in partnership with communities and service providers around Australia has been addressing these problems.

RSTO provides a framework of evidence-based lead indicators, resources and tools to support service providers (including frontline practitioners), and policymakers to improve service quality, quantity and participation across core early years services.

RSTO focuses on five fundamental early years services: antenatal care, sustained nurse home visiting, early childhood education and care, parenting programs and the early years of school. These five services each have a sound evidence base that shows they are beneficial. RSTO’s insights have relevance for other social sectors that are working to address inequities. The article also outlines the key role governments and philanthropists can play to support service providers and communities to be better equipped to collect and use data to support systems changes.

Key challenges

The following are the key challenges in the current approach to data use.

1. Why we can’t tell what’s happening from the data we have now

Most current measurement frameworks rely heavily on outcome or ‘lag’ indicators. While necessary for evaluating population level progress, lag indicators are insufficient for driving timely and effective changes on the ground. The long delay – sometimes years – between action and outcome measurement makes it nearly impossible to determine if actions are effective in real time.

These types of measures don’t provide information about how improvement could or should be made and don’t discriminate between good and bad practice.

Lead indicators (or ‘process’ indicators), on the other hand, more closely reflect the activities mostly directly undertaken by and in the control of frontline practitioners. They are easier to interpret and help in adjusting responses. (See Table 1.)

Without knowing what services children and families receive, it’s impossible to assess what leads to better outcomes and where to focus effort. Timely information is critical to both empower frontline workers and communities to assess performance and progress regularly, enabling a better understanding of where to target priorities, actions and investments.

2. Lack of consensus on what to measure

There is currently no common framework across Australia that sets out what the lead indicators across early years services should be. There aren’t indicators that set out what we expect every child and family to receive, how much is the right amount, or what good looks like.

Typically we see service providers and communities struggling to understand what data would inform decisions. We see the one-off development of metrics and methodologies, significant effort debating what to measure, and puzzling over masses of unhelpful data that take a long time to get consensus on.

These approaches limit the opportunity to share data and what services have learned and inhibit their ability to tailor solutions that better meet the needs of children and families.

3. Service providers aren’t given enough support to use data

Through work with partners, RSTO identified that while the data they need typically exists, it is often inaccessible, trapped within individual organisations’ databases or even in paper forms. Extracting this data securely is slow and time-consuming, hindering momentum.

Furthermore, even if the right data were available, many organisations do not have the resources, skills and routines to execute a regular cycle of measuring, analysing, course-correcting, and re-measuring – what we refer to as continuous improvement.

Service providers and other decision-makers require real investment and support to work in a culture of continuous improvement. They need both access to easy-to-interpret data and a supportive environment that fosters using data, with dedicated time and resources to reflect on data and collaborate on solutions.

4. Limited insight on incentives to use data

RSTO found there is little understanding in early childhood sectors of what motivates routine data use.

Generally, people are encouraged by extrinsic motivators (e.g. more funding to do innovative work) and intrinsic motivators (e.g. doing something because doing it makes them happy or benefits someone). Service providers may be required by funding agencies to collect large amounts of data as part of service reporting, or to support outcomes measurement initiatives. However, this data is typically not shared back to the service providers by the funding agencies in a format that is meaningful or motivates use. m.

RSTO’s research has revealed both extrinsic and intrinsic are critical to both understand and address in order to build routine use of data in decision making.

How RSTO is addressing these key challenges

RSTO is a collaboration between Murdoch Children’s Research Institute, Social Ventures Australia, and Bain and Company. Over the last several years, we have partnered with early childhood service providers and communities across Australia to understand these challenges at a practice level and co-design solutions to address them.

Our work has demonstrated that:

1. It is possible to apply a common data framework

RSTO has defined evidence-based lead indicators that can be used to assess the performance and progress of the five fundamental early years services: antenatal care, sustained nurse home visiting, early childhood education, parenting programs and the early years of school.

What we want to measure is whether children are receiving high quality services when and where they need them.

The simple three-part structure RSTO has developed covers:

• Quantity: Are the services available relative to the size of the target population?
• Quality: Are the services delivered effectively relative to evidence-based performance standards?
• Participation: Do children and families participate, and at the right frequency?

Importantly, this framework is solution agnostic, meaning it can be applied consistently across services, yet provides actionable insights for each. This approach could also be easily applied to other evidence-based services such as housing.

Also, this data can be coupled with other data collected by communities and service providers such as community voice or client satisfaction surveys. Combining this information ensures actions in response to the data are tailored appropriately to meet the needs of a specific population or community.

2. Tools for consistent collection of lead indicators, interpretation and sharing can be developed

RSTO has co-designed an easy-to-use data platform with service providers and community partners.

This platform allows for:

• secure uploading of de-identified data from service systems;
• standardised indicators and data structures that allow comparison longitudinally and across service delivery locations and organisations;
• automated calculations that translate input data into indicators;
• common-sense visualisation tools co-designed with end users; and
• standard privacy protocols and data security.

Partners have told us that when they are able to see their data in one location, it is much easier to see where their efforts are succeeding.

3. Investment is needed to build capability for data use

RSTO has developed a continuous improvement program that supports frontline workers and community to use data efficiently and effectively.

An RSTO continuous improvement coach meets with practitioners semi-regularly to go through their most recent data, helps them understand it and guides them to think about what actions they might take in response to the data, using a simple plan, do, study, act cycle. Plans are agreed, tested and then data is monitored for change.

RSTO’s work has uncovered that investment in resources to support continuous improvement is essential to drive action. Over time, as capabilities are developed and practices embedded the amount of support required reduces.

4. The incentives needed to drive data use and action

RSTO has identified intrinsic and extrinsic motivators that have successfully supported early childhood frontline practitioners to adopt and use data effectively.

RSTO’s research and on-the-ground practice found a commitment to multi-year investments in resources to support data use together with funding for local solutions identified through data are essential incentives.

We also found recognising the efforts of frontline practitioners providing the services is important. If they know that their work makes other people’s lives better, they’re more likely to go the extra mile, and to willingly adopt the daily practices that enable data collection.

Finally, involving practitioners in solution design, as part of a continuous improvement process, is a critical motivator, empowering them to test, try and monitor ideas quickly.

Moving forward

From the outset, RSTO have received strong support for its work and its aspiration. Experts in each of the five services are frustrated when funds aren’t directed efficiently and committed to projects without a sound understanding of the community they are targeting or practical tools to measure their impact.

Service providers, government policy-makers and community representatives recognise the potential opportunity the RSTO framework, tools and resources present to inform purposeful improvements targeted to families who need these supports the most.

We have been encouraged to see early signs of action in communities and service providers who have pioneered this work with us.

To foster measurable and actionable system change, it is imperative to develop the capability to harness and use lead indicators effectively. ”

Gladstone Region Together, another place-based initiative, is also partnering with RSTO, recognising the unique perspective that the restacking data brings to supporting services in the region and to also identify where their effort should prioritised.

Gowrie Victoria, uses RSTO across five Victorian early childhood education centres to understand children’s participation patterns and to help identify local solutions to increase participation.

Bourke and District Children Services in NSW used RSTO data to help understand how many children are missing out on access to early childhood education and care in their community and to advocate to government for additional places.

To foster measurable and actionable system change, it is imperative to develop the capability to harness and use lead indicators effectively.

Governments and philanthropists can support this effort by

1) adopting consistent, evidence-based lead indicators that set out what we expect every child to receive, how much, how often, and what good looks like

2) investing in infrastructure and resources to support collection and ongoing use of that data.

By simplifying data collection processes, and encouraging the use of data for continuous improvement, we can create a more equitable and effective early childhood development ultimately improving life trajectories for children born into disadvantaged circumstances.

Authors: Olivia Hilton and Jacqui McCann