Fairest and best? Status counts in the Brownlow Medal

Tonight is the AFL’s annual night of nights, the red-carpet spectacular known as the Brownlow Medal vote count.

The Brownlow is awarded to the season’s “fairest and best” player. But is the way the medal gets decided really the fairest?

Research I’ve done with Aaron Smith of Loughborough University suggests footballers don’t compete for the Brownlow on a perfectly level playing field. There’s a slight bias towards team leaders.

As a sports economist, I’m fascinated by what the Brownlow Medal count can tell us about the science of evaluating performance in any organisation. In any job, all other things being equal, a performance appraisal may be better (all else equal) once you have worked your way up the corporate ladder. In a perfect meritocracy this would not be the case.

Evaluation bias

Aaron and I have long been interested in the extent to which people in leadership positions receive undeservedly high performance evaluations just because of the managerial networks that arise from the status those positions provide.

Generally we think this occurs because of inevitable human-relationship factors. The more interesting question is whether it happens in cases where the evaluation process is entrusted to external agents (such as HR consultants) supposedly independent of such influences.

This is a difficult question to investigate. Business organisations are not inclined to hand over employee evaluation data to pesky academics. The Brownlow offers a rare analogous opportunity to analyse empirical data.

Brownlow Medal votes are allocated by field umpires (who are, of course, similarly “external” to the competing firms). After each game of the regular season they confer and nominate the three top players. The best player gets three votes, the second two and the third one. This is actually a very rare case within the pantheon of global best-player awards in major sporting leagues where umpires cast the votes.

And come medal count night, the votes are in the public domain.

The data goes back to 1984 (before that, votes were not recorded to the match in which they were cast), and is available on the AFL Tables website. We have used different statistical tools to analyse the data, as well as a comprehensive database of match statistics for about 100,000 player-within-match records from all 2,254 home-and-away AFL matches played from seasons 2006 to 2017. These statistics cover the number of goals, kicks, handballs, tackles and so on by each player.

Significant statistics

Testing for bias towards team leaders has an intuitive underlying justification. Umpires have more contact with captains, such as through the pre-match coin toss. This may unconsciously influence umpires’ perceptions.

The raw data did tell us captains earn, on average, a lot more more Brownlow votes. But this alone does not imply a voting bias. Other factors must be considered. For one thing, captains are likely to be credentialed and talented players. That’s why they’re captains.

We sought to take this into account by figuring into our model a large range of player statistics indicating individual performance within each match.

Our results show that, when controlling for player performance, team captains poll between 1.3% and 2.1% more votes than all other players. There is a strong statistical significance here.

Read more: Mind your confidence interval: how statistics skew research results

The effect for vice-captains was smaller, from 0.6% to 1.5%. There was no significant vote boost for acting captains who normally have no leadership position.

Mitigating factors

In explaining the results, we cannot rule out possible actions not captured within our statistics at which captains have excelled and that have been noticed by umpires.

It may be, for example, that captains have kicked a crucial goal with a higher than average degree of difficulty or made a decisive tackle that has helped turn a game. Statistics alone don’t tell the whole story. Identical player data doesn’t necessarily have the same vote “impact”.

But based my own experience as a former umpire for 15 years (at suburban level for Melbourne’s Northern League), I am convinced captaincy status does really matter in junior games when the umpires are often not previously familiar with any of the players.

In previous research I’ve identified similarly significant umpire biases when choosing “fairest and best” players, such as in favour of Indigenous players and those playing milestone matches.

Field of favourites

But it’s a game of percentages. No captain has won the Brownlow since Carlton’s Chris Judd in 2010. Essendon captain Jobe Watson did poll the most votes in 2012, but he was later stripped of the award due to the club’s doping scandal.

Read more: The Japanese art of kintsugi and how it can help with defeat in sport

So I am noncommittal about tonight’s Brownlow Medal outcome, though our research does seem especially relevant given the favourites this year.

Most obviously there is Carlton’s Patrick Cripps, recently awarded the Leigh Matthews Trophy given by the AFL Players Association to the league’s “most valuable player”. Cripps also became Carlton co-captain this year.

Also among the bookies’ favourites (though at longer odds) are Fremantle captain Nathan Fyfe and Collingwood captain Scott Pendlebury.