Blogging another inquiry: Valuing the Australian Census

Lateral Economics has been commissioned by the Australian Bureau of Statistics (ABS) to estimate the value of the Australian Census to the Australian community. As part of that exercise we’ve got the go-ahead from ABS to do something that, it seems to me, all public inquiries, and most independent public agencies, should do as a matter of course. Run a blog a little like the blog we ran when I chaired the Government 2.0 Taskforce.

Ten years on from the Taskforce, it’s amazing how slow the practice has been to catch on. It seems to me the kind of thing that the Productivity Commission should do, and the Reserve Bank. Anyway, as far as I know, there’s no such blog on offer there or anywhere else much. One exception is the Bank of England which runs a lively blog in which staff explore all manner of questions relating to their research. It greatly enhances both its work and their staff’s morale and willingness for some of the best to stay at the Bank rather than moving off into academia or consultant land.

Anyway, this exercise gives us an opportunity to trial it again. Accordingly we will post from time to time outlining the progress of our investigations, particularly highlighting issues we are trying to understand to improve the quality of our review.

Comment will be post-moderated with a back-up plan to provide pre-moderation in the unlikely event of commentary which is judged to lack bona fides in a disruptive way. We’ll be consulting ABS with draft blog posts before posting them to give the ABS an opportunity to provide input. We nevertheless remain responsible for the exercise, and for the content of each post, not the ABS.

Finally, if you do have anything to offer, please do so to show how productive this kind of exercise can be. I know myself how productive it is from my own experiences blogging myself not least as part of the Government 2.0 Taskforce, but as I’ve grown older it’s struck me how much adoption relies on, well, adoption. If no-one’s doing it, no one does it. Then a few people do it and the obviousness of doing it seems to become accessible to others, even as they keep their in-tray reasonably cleared.

Valuing the Australian Census

Gene Tunny, Nicholas Gruen, and Matt Balmford, Lateral Economics

The five-yearly Australian Census provides a wealth of data on Australian households. Those data are used extensively by all levels of governments, industry, and non-profits in decision making and planning. The Census provides important information on household composition, journey-to-work flows, Indigenous status, and languages spoken at home, among other data. It is also used extensively to ‘ground-truth’ and so assess and improve the representativeness of a wide variety of data sets. The ABS understands that all this information is of substantial value, but no careful estimate of its total value has yet been made. The ABS has engaged Lateral Economics to provide such an estimate.

In recent years, a combination of factors has led to renewed scrutiny of the value of the census, including burgeoning administrative data which could be tapped by national statistical agencies, as well as advances in computing power and data science. Statistical agencies worldwide have come under pressure from ministers to provide greater justification for the hundreds of millions of dollars of costs of a national census. For instance, in 2001, the UK House of Commons Treasury Select Committee called for the national census to be justified in cost-benefit terms. Since then, the UK Office of National Statistics (ONS) has produced a Census Benefits Evaluation Report for the 2011 Census. It was reported that the UK census yields annual benefits in the order of £500 million.

In 2013, Statistics New Zealand followed the example of the ONS and commissioned a valuation of the NZ census, which reported a return on investment (ROI) of around $5 of net benefits for every $1 of expenditure on the census.

In Australia, too, the issue of the cost of the census has come under scrutiny. For example, in 1993, a Federal government interdepartmental committee investigated cost reduction opportunities for the census. However, it recommended maintaining the current census format. Indeed, comprehensive five-yearly censuses are seen as necessary to ensure fair electoral representation given substantial population growth in newly-developed areas, such as Springfield and Yarrabilba in South East Queensland. The requirement for a five-yearly Census was inserted into the Census and Statistics Act 1905 in 1977 (section 8(1)), following a 1976 High Court decision suggesting the need for more a more frequent Census to ensure fair electoral representation.

A critical question for Lateral Economics’ research project is the extent to which the Census is essential for developing high quality estimates of populations at various levels, national, state, regional and for small areas, and for various groups in the population, including Indigenous and culturally and linguistically diverse people. Such estimates are important for electoral representation, but they are also important for the allocation of:

  • GST revenues across state and territories;
  • resources in health, education, and other social services; and
  • capital expenditures, both for social infrastructure and transport infrastructure, and in the private sector, too (e.g. shopping centres, retirement villages, etc.).

Such decisions may depend critically on information such as accurate estimates of Indigenous people in a region, or journey-to-work flows between regions, and the Census is a critical source of information to such estimates and for which there are not obvious substitutes.

Lateral Economics will attempt to quantify the benefits to the Australian community of more accurate decision making regarding the allocation of funds across regions for different purposes. We will be guided, but not limited, by the methodologies employed in previous studies. These methodologies rely largely on estimates of the inaccuracies of resource allocation and capital investment that would be occur in the absence of Census data. We will also draw on methodologies developed in these studies for estimating the dollar benefits of, among other things:

  • improved public policy development in social policy areas where Census data are important, such as in policies for disadvantaged groups1; and
  • higher-quality academic and market research.

Even after the most stringent of efforts, considerable uncertainties will remain in parameterising these questions. However this underlines the importance of consulting as widely as possible – with the Australian Electoral Commission, Commonwealth Grants Commission, other federal and state agencies, local governments, and the private sector and NGOs – to develop defensible assumptions to underpin our estimates (guesstimates?). It is likely we will provide reasonable lower and upper-bound estimates of this ROI.

In undertaking this project, Lateral Economics is not necessarily presuming that there will net benefits to the community from running the Australian Census in its current format and frequency. We are doing our best to follow the evidence – something that the ABS has stressed its support for. We note that, based on a cost-benefit analysis, South Africa decided not to run a census in 2016, so the question of whether a national census is valuable is an open one.  But wherever the evidence takes us, it’s critical we get the best evidence we can regarding users’ specific uses of Census data, the benefits they derive from that use, and any reflections they have on what the next best source of data would be in the absence of the Census, and the potential costs of any reduction in data quality.

In future articles, Lateral Economics will discuss our progress and seek feedback on our thinking and analysis to date regarding the different Census benefit values we will be estimating. In the meantime, if you have any ideas or information that you think would help us, please get in touch via mail@lateraleconomics.com.au.

1 This could be estimated by assuming a percentage improvement in the quality of related government services and applying that to total spending on those services. The assumption would be informed by consultations with stakeholders.

  1. 1[][]
Subscribe
Notify of
guest

25 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
paul frijters
paul frijters
5 years ago

jeez, what a question, ‘the exact value of the census’. How on earth are you going to try to do that? The census is part of a whole system of self-knowledge that could be used by a competent and pro-social bureaucracy and civil society. The census is the show-piece of the statisticians, a regular production to keep them employed and practiced so that there is data on lots of things.

It is a bit like a sense of smell, hearing, feeling, or sight. Individuals who lack one of their senses can be reasonably functional, but clearly less than they could be. Countries without a census (and who also dont have a population register, which is actually much better) are very few and usually war-torn.

One might try to get at it indirectly via the value of good governance and then plug in some guess on the part of good governance that is good statistics, and then the part of the census in good statistics.

Of course, the inappropriate secrecy procedures of the ABS (which I believe violate their legal obligations under the Act that regulates them to use the data for the benefit of all Australians using outside help as they see fit) greatly reduces the value of all their data. The value of the census is a lot lower under the current conditions of inappropriate secrecy than it was before. By their secrecy the ABS has damaged itself needlessly the last few decades.

However one generates a number though, it will be extremely speculative. That is true for many other things too, like defense or national heritage.

I wonder why they need a number enough to pay for one. Is anyone suggesting to privatise the census or the ABS and do they need a number in self-defense?

Gene Tunny
Gene Tunny
5 years ago
Reply to  paul frijters

Paul, great comments, thanks. No, no one is suggesting privatising the Census at this stage, or scrapping it entirely. There have been debates in the past about whether it could be run less frequently and with fewer questions, though. I don’t think there’s an immediate threat that would warrant commissioning this study in self-defense. I think it’s more about wanting evidence that can help them communicate the benefits of the Census to the public in the lead up to the 2021 Census so people fill it out conscientiously and accurately.

Yes, you’re right the benefits estimate will be extremely speculative, and we expect we’ll need to a have wide range around it.

paul frijters
paul frijters
5 years ago
Reply to  Gene Tunny

Gene,

you were the one who wrote that doubtful piece on my book “game of mates” that took on the various rent-seeking schemes across Queensland and Australia where we generated an estimate of the full cost of it.

Your use of the word ‘we’ suggests you’ll be charged with coming up with just such a number, this time for the value of the census? The irony :-)

conrad
conrad
5 years ago

I don’t see why you need to run census like surveys anymore. If you can find out 99% of what you need by voluntary sampling, then unless you can justify the 1% I don’t see why you need them. I’d personally like to see a list of things that you really need such a massive and invasive survey for where the tiny accuracy difference actually made any difference to policy ever.

As a side note, at present the Australian one is especially annoying because the information site says you can voluntarily participate, but it isn’t voluntary at all (like the similar household one which you can get to involuntarily be a part of too). So it is outright dishonesty — I complained and asked them to change their website so it was honest to no avail.

Also, if you don’t fill it in, a person will come around to your house and try and force you to (with no real ID). Apparently you can get fined for not filling it in. In these situations, people will make junk up and smarter ones that resent this government sledgehammer approach will make up junk 2.99 SDs from the centroid so it is difficult to tell if it is junk and remove. Given such people are probably some percentage of the sample, voluntary sampling may well provide more accurate results.

paul frijters
paul frijters
5 years ago
Reply to  conrad

I dont care so much about that particular dishonesty because there is a sense in which this should just be compulsory for the same reason as voting. However, if we are going to see the data of the population as useful for the public good, they should bloody well indeed then use it, but they dont because of the political fear. They dont feel under real pressure to be open with their data, so I do think in principle they can just as well offer a private firm and another country to do it for less.
I am also not too fussed by the cost. A census is no longer needed for tax purposes, but its handy to round the houses and look at all the places population might be hidden. Besides, there is also a point in having a few spare statisticians. It’s the cowardice with regards to public usage that I find the most annoying. It is not really their data but they act like it is.

paul frijters
paul frijters
5 years ago
Reply to  Nicholas Gruen

“The ABS has an obligation imposed from without”

I dispute that. I read the relevant Act on the “Australian Statistician” and it seems to me they can just as well read that to mean they should enlist help whereever they can under conditions they get to set. The super-secrecy stuff is self-imposed. The reasons for it are internal, not external.

Its highly relevant to the post because it affects the value of the census. Via a simple change of mindset they could increase that value tremendously. Besides, blogs are meant to get people’s mind wondering a bit. Not too far, but certainly a bit.

paul frijters
paul frijters
5 years ago
Reply to  Nicholas Gruen

“would protect people’s privacy as much as they’re protected now – or in practice nearly as much.”

but that is the BS. What privacy is respected in an Australia of raids, secret surveillance, hundreds of internet firms that know boatloads about everyone, etc.? To accept that premise is to accept nonsense. I do know that this is the figleaf some ABS people put out, but taken literally they indeed then can give out the unit record data and rightfully claim they are respecting privacy just as much as other agencies and private firms. If the rest of government and private business do not respect privacy, why should the ABS when it’s act does not require that?

I think they should do a Denmark.

Your question is a bit like the question of the value of a sumo wrestler in a competition where he is not using his hands but everyone else is. I keep bringing up how much higher his value would be if he could use his hands like everyone else. You keep saying he needs to stick to rule of using his hands just as much as everyone else. Precisely my point. Then you say someone will criticise him for it. I say coward and that a sumo wrestler is not a sumo wrestler without his hands.

conrad
conrad
5 years ago
Reply to  Nicholas Gruen

I don’t doubt there is some use in the census, and that depending on how you want to measure it, you will find a big benefit. Some of that will always sound like magic to people (e.g., “a 1% efficiency gain on 10 billion dollars of spending”) even if it isn’t (e.g. “a 1% improvement on obesity rates compare to a control”).

So the real questions to me is to what extent the census provides information over other sources you could get easily or already have (you already note this as a possible in your post), and if it does, to what extent you need it. For example, the ABS collects any number of obviously useful bits of information that historically would have been harder to get, e.g., prisoner numbers and their demographics. These are not in the census because they are too detailed, and like many questions, you often need detailed information and a census with very limited questions won’t provide this. Things like prisoner demographics are important beyond just planning for prisoners, but social debate too because if you look through the numbers you find out that the biggest problem is being young and male, not ethnicity. I’ll let you quantify the worth of social debate!

A second problem is many (most) questions don’t require exact information. For example, it doesn’t matter if 10% or 30% of Australians are obese, because having 1% less obese people would be worth a lot. So there is no point in collecting the specific data. Similarly, even with population figures, which is supposedly one of the examples where it is useful, you don’t need perfect information. For example, if I need, say, hospitals or schools, I can get by with sampling (which at a guess would not be very different in error to the census, especially after the recent computer problems putting people off filling it in). Most of these decisions are largely reactionary anyway – although perhaps they could be better if people planned better, which would be an advantage of the current situation, assuming no other information sources.

There are examples here the ABS uses like number of homeless people where they suggest there are low numbers and you really do need the census (unlike more cynical people like me). But it is unclear to me that the sampling here is better than the census, and even if it was, most of the decisions at this level don’t use the ABS data – they’re done by NGOs and since there is never enough funding, even if they were done by government, the problem isn’t knowing the numbers, the problems are social ones and people not wanting their money to go on homelessness. For example, if we didn’t collect homeless numbers in the census, would anything differ policy-wise? If the answer here is no, clearly that aspect of the census wasn’t worth anything except for social debate. A similar example would be numbers of indigenous Australians. Perhaps this is more useful, but we don’t need an entire census to find out.

So to me this leaves the main benefit of a census data as things which then inform other decisions to collect more data to make better decisions. This is far harder to quantify, but if I was the head of the ABS, I’d be telling all the government organisations that use their data to give some acknowledgement that they used it, as, for example, most research and charity organisations do. In this way, you would really be able to say that this was used for something useful in a way that other sources were not able to provide and you could probably have a decent hope of quantifying the benefit even to the disbelievers.

conrad
conrad
5 years ago

You can see it is still dishonest:

https://www.abs.gov.au/websitedbs/D3310114.nsf/home/survey+participant+information+-+Monthly+Population+Survey

This sort of thing wouldn’t pass a standard ethics committee so I don’t see why the ABS should be allowed to do it given it has no reason to be dishonest.

Why not just say: “You are obliged to do this, or you will be fined”

paul frijters
paul frijters
5 years ago
Reply to  Nicholas Gruen

“completely antithetical to the political values of the country (for all their inconsistency), to take an interpretation of the words of the Act that would do no more than cause uproar before being immediately overturned in parliament if it couldn’t be done by ministerial direction”

all that is just not true and a matter of how things are framed. Doesnt sound like you either. Are you copy-pasting from someone else?

Australia is known in statistics land to be an outlier in the OECD, an aberration. Even the US is not like Oz. The ABS should simply explain that they indeed want the level playing field with business and other parts of government.

All this BS is basically taken as an excuse to spend lots of government money on data which it then bars the research community from using properly.

John R Walker
5 years ago

Curious why the ABS has commissioned research into the value to Australia of the census data in the first place, is the census under some kind of threat?

conrad
conrad
5 years ago
Reply to  John R Walker

The census is always under some kind of threat at least in terms of how often it gets run. Historically, this wasn’t so bad, because the baseline was essentially no data, so you could argue that it was of high value no matter what. However, one could argue that the baseline now is collection via other cheaper and possibly more accurate methods, and so the value is probably the difference between the ABS methods and alternative ones. This is why the ABS went to seemingly great effort to highlight data that might otherwise not be collected. However, as Paul notes, it’s actually very hard to get access to it, so its value is more limited.

It is worthwhile noting that the ABS is also under political threat — a large chunk of their staff were forced to move to Geelong for largely political reasons recently (the move may not have happened yet), and so you can imagine they are pretty desperate to highlight the job they do, especially after their computer problems, which look bad to the public.

I personally think the ABS should be treated like the RBA — they should be independent, and they could provide real data versus the type of mush you get from organisations more under the thumb such as government departments and all of the agencies that they contract to provide reports that always put them in a good light. It is very hard to put a dollar value on this that people might believe as being somehow correct (perhaps low-and-high estimates could be done — I’ll leave that to Nicholas :) ), but like the RBA, they clearly have great value compared to politically likely alternatives, and I suspect that most people that understand what they do would probably agree with that.

conrad
conrad
5 years ago
Reply to  conrad

Now I think about it, here is an example of why the ABS should release its data that would show why it is better than the alternatives (you could probably dig this up from census data if you had the full set, which is not released). This way it could claim to really be an independent marker of the truth, and not a data hoarder trying to please the government — their data collection would also be of much greater value.

Let’s take a topic like health. Some areas are always controversial like Aboriginal health, but the outcomes are very heterogeneous. So the first thing that will happen is that you will ask your data processor to simply use one category “Aboriginal”. This still might look bad, so you then decide that this category should be deleted altogether, so you now dissolve it into low SES. This now drags you low SES category down, so you rejig the percentiles use (Bottom 5%? Bottom 10%, bottom 20%) etc. . So this way no-knows the problem, and no-one can find out.

A real dataset just came out in Victoria with a similar sort of finding — basically country kids are doing worse in school than kids from the city. This is no surprise, but the results must have been around for ages — so they simply haven’t released and everything averaged away like my example above. So the Andrews government is at least better than previous ones on this. I’m willing to be that the kids in really remote places are largely responsible for this, but alas, you can’t get that data, and no government wants to think of solutions for this. So everything is still hidden in a way that it shouldn’t be.

John R Walker
5 years ago
Reply to  conrad

Thanks. Yes its the areas of strategically placed ‘blur’ that often matter much more than what can be seen clearly.

Wilson Pink Trewin Coleman
Wilson Pink Trewin Coleman
5 years ago

The Census is – at best – the standard you hope to beat. At worst, it is a nearly half billion imposition on the community spread over five years with pointless lags and embeds incorrect static estimates.

The most important output is local population estimates, but the only reason this is important is other legislation: e.g, local area estimates to determine funding, electoral boundaries etc.

Census results are always one year or more out of date by the time of publication.

Admin data can already provide up-to-date estimates of the age profile, and are pretty close to current meshblock estimates.

George Bray
George Bray
5 years ago

Well that’s a good project to work out the value of the census. I suspect the LNP/IPA are gathering evidence for outsourcing it to their mates.

Last time, the sniff of possibility that my data would be sold to their mates had me not participating, energetically. My stance stands. I now don’t trust the process and will not be providing my data to this regime until evidence is received that my privacy is protected.

I’m not sure if this is of any use to you, but there are many in my IT and privacy circles that have seen the line crossed. From my perspective the value of the census is now zero.

John R Walker
5 years ago
Reply to  Nicholas Gruen

Nicholas
While not questioning the arms length approach viz you and your colleagues .

As a tax payer I’m not keen on any publicly funded body paying for research ,into the ongoing value of what it’s funded to do.

Matt Cowgill
5 years ago

On institutional blogs, the relatively new Grattan Blog may be of interest: https://blog.grattan.edu.au