Modern Australian
Men's Weekly

.

Robodebt not only broke the laws of the land – it also broke laws of mathematics

  • Written by Noel Cressie, Distinguished Professor of Statistics, University of Wollongong
Robodebt not only broke the laws of the land – it also broke laws of mathematics

Friday marked the end of the public hearings for the Royal Commission into the Robodebt Scheme. They painted a picture of a catastrophic program that was legally and ethically indefensible – an example of how technological overreach, coupled with dereliction of duty can amount to immense suffering for ordinary people.

The artificial intelligence (AI) algorithm behind Robodebt has been called “flawed”. But it was worse than that; it broke laws of mathematics. A mathematical law called Jensen’s inequality shows the Robodebt algorithm should have generated not only debts, but also credits.

What was Robodebt?

The Australian government’s Robodebt program was designed to catch people exploiting the Centrelink welfare system.

The system compared welfare recipients’ Centrelink-reported fortnightly income with their ATO-reported yearly income, the latter of which was averaged to provide fortnightly figures that could be lined up with Centrelink’s system.

If the difference showed an overpayment by Centrelink, a red flag was raised. The AI system then issued a debt notice and put the onus on the recipient to prove they weren’t exploiting the welfare system.

A Robodebt victim

To understand the extent of the failure, let’s consider a hypothetical case study. Will Gossett was a university student from 2017-2019. He was single, older than 18, and living at home with his parents.

Will received Centrelink payments according to his fortnightly income from a couple of casual jobs with highly variable work hours. In his first year at university his jobs didn’t pay much, so he received more Centrelink payments in the 2018 financial year than the year following.

The Robodebt algorithm took Will’s ATO yearly income records for both the 2018 and 2019 financial years and, for each year, averaged them into a series of fortnightly “robo” incomes.

Inside Robodebt’s AI world, his fortnightly incomes were then the same throughout the 2018 financial year, and the same throughout the 2019 financial year.

Will was honest with his claims, but was stunned to receive a debt notice for Centrelink overpayments made in the 2019 financial year – the year in which he actually received lower welfare payments.

The income-averaging algorithm gave Will an average fortnightly income for 2019 that was above the threshold that made him eligible for Centrelink payments. As far as the Robodebt system was concerned, Will shouldn’t have received any welfare payments that year.

Read more: 'Amateurish, rushed and disastrous': royal commission exposes robodebt as ethically indefensible policy targeting vulnerable people

Jensen’s inequality

The laws of mathematics tell us when two things are equal, but they can also tell us when one thing is bigger than another. This type of law is called an “inequality”.

To understand why and when Robodebt failed for Will, we need to understand a concept called Jensen’s inequality, credited to Danish mathematician Johan Jensen (1859-1925).

Jensen’s inequality explains how making a decision based on the averaging of numbers leads to either a negative bias or a positive bias under a “convexity condition”, which I’ll explain soon.

You’ll recall Will is a single university student, above 18, and living with his parents. Based on these factors, Centrelink has a fortnightly payment table for him, illustrated with the curve in the figure below.

The figure shows the more income Will earns from his jobs, the less welfare payment he receives, until a specific income, after which he receives none.

This graph, created from tables provided by Centrelink, shows how certain factors determine the amount of welfare payments Will can receive depending on his income.

The parts of the curve where Jensen’s inequality is relevant are highlighted by two red squares. In the square on the left, the curve bends downwards (concave), and in the square on the right it bends upwards (convex).

Because Will’s income was higher in 2019 and spread across the part where the payment curve is convex, Jensen’s inequality guarantees he would receive a Robodebt notice, even though there was no debt.

In 2018, however, Will’s income distribution was spread around smaller amounts where the curve is concave. So if Jensen’s inequality was adhered to, the AI algorithm should have issued him a “Robocredit” – but it didn’t.

It could be the algorithm contained a line of code that nullified Jensen’s inequality by instructing any credits be ignored.

Big data and a bad algorithm

The people responsible for the Robodebt system should have had a strong interest in keeping error rates low. Data scientists have a big red “stop” button when error rates of automated systems go beyond a few percent.

It’s straightforward to estimate error rates for an AI scheme. Experts do this by running simulations inside a virtual model called a “digital twin”. These can be used to carry out statistical evaluations, and expose conscious and unconscious biases in bad algorithms.

In Robodebt’s case, a digital twin could have been used to figure out error rates. This would have required running the Robodebt algorithm through representative incomes simulated under two different scenarios.

Under the first scenario, incomes are simulated assuming no debt is owed by anyone. Every time a result is returned saying a debt is owed, a Type 1 (or false-positive) error is recorded. Under the second scenario, incomes are simulated assuming everyone owes a debt (to varying degrees). If a no-debt result is returned, a Type 2 (false-negative) error rate is recorded.

Then an error rate is estimated by dividing the number of errors by the number of simulations, within each scenario.

Eye-watering inaccuracies

Although no consistently reliable error rates have been published for Robodebt, a figure of at least 27% was quoted in Parliament Question Time on February 7.

The reality was probably much worse. During the scheme, on the order of one million income reviews were performed, of which 81% led to a debt being raised.

Of these, about 70% (roughly 567,000 debts) were raised through the use of income averaging in the Robodebt algorithm.

In 2020, the government conceded about 470,000 debts had been falsely raised, out of a total of about 567,000.

Back-of-the-envelope calculations give a Type 1 (false-positive) error rate on the order of 80% (470,000/567,000). Compared to the usual target of a few percent, this is an eye-wateringly large error rate.

If simulations had been run, or human intelligence used to check real cases, the “stop” button would have been hit almost immediately.

Jensen’s inequality establishes why and when income averaging will fail, yet income matching hasn’t gone away. It can be found in AI software used for official statistics, welfare programs, bank loans and so forth.

Deeper statistical theory for this “change of support” problem — for example, going from data on yearly support to fortnightly support — will be needed as AI becomes increasingly pervasive in essential parts of society.

Read more: Why robodebt's use of 'income averaging' lacked basic common sense

Authors: Noel Cressie, Distinguished Professor of Statistics, University of Wollongong

Read more https://theconversation.com/robodebt-not-only-broke-the-laws-of-the-land-it-also-broke-laws-of-mathematics-201299

How To Keep Vase Flowers Fresh Through Australia’s Coldest Months

Winter flowers develop slowly, which gives them stronger structure and longer vase life Heat from indoor environments is the biggest threat to th...

Artificial Intelligence is Powering the Growth of Australian Telehealth Services

Many Australians have traditionally experienced difficulties in accessing timely and quality healthcare, especially those who live in rural or remot...

Powering Shepparton’s Businesses: Expert Commercial Electrical Services You Can Count On

When it comes to running a successful business, having reliable, compliant, and efficient electrical systems is non-negotiable. From small retail ou...

Maximise Efficiency: Cleaner Solar Panels for Optimal Performance

Solar panels are a smart investment in energy efficiency, sustainability, and long-term savings—especially here in Cairns, where the tropical sun ...

7 Common Air Conditioner Issues in Melbourne – And How to Fix Them

Image by freepik Living in Melbourne, we all know how unpredictable the weather can be. One moment it’s cold and windy, the next it’s a scorchin...

Powering Palm QLD with Reliable Electrical Solutions

Image by pvproductions on Freepik When it comes to finding a trustworthy electrician Palm QLD locals can count on, the team at East Coast Sparkies s...

The Smart Way to Grow Online: SEO Management Sydney Businesses Can Rely On

If you’re a Sydney-based business owner, you already know the digital space is crowded. But with the right strategy, you don’t need to shout the...

What Your Car Says About You: The Personality Behind the Vehicle

You can tell a lot about someone by the car they drive—or at least, that’s what people think. True Blue Mobile Mechanics reckon the car says a l...

The Confidence Curve: Why Boudoir Photography Is the Empowerment Trend You Didn’t Know You Needed

Boudoir photography has been quietly taking over social feeds, Pinterest boards, and personal milestones—and for good reason. It’s not just abou...

Understanding Level 2 Electricians: Why Sydney Residents Need Licenced Experts for Complex Electrical Work

When it comes to electrical work around the home or business, not all electricians are created equal. In Sydney, particularly when you're dealing wi...

Retirement Anchored in Model Boat Building for Waterford’s Doug Unsold

WATERFORD — When Doug Unsold sees his ship come in, it’s usually one he’s crafted with his own hands. The 67-year-old retiree from Waterford ...

The Science Behind Alarm Clocks and Your Circadian Rhythm

Waking up on time isn’t just about setting an alarm—it’s about working with your body, not against it. At the heart of every restful night and...

How to Use Plants to Create a Calming Atmosphere in Your Home

In today’s fast-paced world, cultivating a calm, soothing environment at home has never been more important. Whether you live in a busy urban apar...

How Maths Tutoring Can Help Students Master Maths

Mathematics can be a daunting subject for many students, often causing stress and frustration. However, maths tutoring has proven to be an effective...

Refurbished iPads Are Better Than New Ones (Here's Why)

Image by rawpixel.com on Freepik Apple's refurbished iPad program has quietly become one of the best deals in tech. While everyone obsesses over the ...

Your Guide to Finding the Right GP: What Perth City Doctors Offer Today

Choosing a General Practitioner (GP) is one of the most important health decisions you’ll make. Luckily, Perth’s vibrant CBD now hosts a new ge...

Why Every Mining Operation Needs a Robust Safety Management System

Mining is one of the backbones of the Australian economy, particularly in Western Australia. Back in 2019-20, mining contributed 10.4% of Australia...

Australian Classic Literature Enjoys Resurgence

Welcome back to the good old days of storytelling! As the modern world becomes increasingly more demanding, returning to childhood favourites offers...