Navigating Australia’s Data Landscape

Government data is increasingly being viewed as a national asset. However, government departments independently manage their own assets and the data landscape can be fragmented and hard to navigate for researchers and policy makers.
Finding the right data to answer policy and research questions often means a lot of legwork and trial and error to make sense of what is available. In many cases, data users “don’t know what they don’t know” – if only there was a solution to help data consumers find the most appropriate data source(s) to meet their needs, and unmask new links with other datasets they’d never thought of?

The ABS is the Australia’s national statistics office and compiles statistics that inform critical decisions across government, businesses and communities. From real-time economic indicators to the nationwide Census, ABS data forms part of a formidable evidence base for decision making across a nation of 27 million people.
In addition to the ABS, data is regularly published by other federal departments, and at the state and local council levels. Whilst this data forms a rich treasure trove of information, the volume and disparate host sites for such information can mean data gets overlooked or underutilised and that information siloes can inadvertently be created.
To bridge this, the ABS is looking for an approach or prototype for creating a searchable "national inventory" of publicly available data that can help users make sense of what is available. You should consider:
- A description of available data assets – what’s in the data? When was it collected? For what purpose? Are their department-specific catalogues already available? Is metadata available?
- Is the data readily available or does it need to be requested? Who is the data custodian?
- How do you get the data? Can it be downloaded? Is there an API?
- How are data sets related to one another? Are they part of series? Is one dataset used to create another?

Your task is to develop a single interface or solution that helps users navigate this environment by dynamically understanding data available from across government. The solution should be explainable and scalable. You should consider the user experience and how they can get what they need using natural language and/or intuitive searching.

In developing an initial solution, consider the ABS and at least two other agencies with significant data holdings, and how the solution creates a simple view across their combined data holdings. Bonus points if you include data from the ABS DATA API, for more information on how to find/use the data, check our User guide: https://www.abs.gov.au/about/data-services/data-explorer/data-explorer-user-guide