Is it actually false, or do you just disagree? Why Twitter’s user-driven experiment to tackle misinformation is complicated

Over the past year, we’ve seen how dramatically misinformation can impact the lives of people, communities and entire countries.

Read more: Public protest or selfish ratbaggery? Why free speech doesn't give you the right to endanger other people's health

In a bid to better understand how misinformation spreads online, Twitter has started an experimental trial in Australia, the United States and South Korea, allowing users to flag content they deem misleading.

Users in these countries can now flag tweets as misinformation through the same process by which other harmful content is reported. When reporting a post there is an option to choose “it’s misleading” — which can then be further categorised as related to “politics”, “health” or “something else”.

According to Twitter, the platform won’t necessarily follow up on all flagged tweets, but will use the information to learn about misinformation trends.

Past research has suggested such “crowdsourced” approaches to reducing misinformation may be promising in highlighting untrustworthy sources online. That said, the usefulness of Twitter’s experiment will depend on the accuracy of users’ reports.

Twitter’s general policy describes a somewhat nuanced approach to moderating dubious posts, distinguishing between “unverified information”, “disputed claims” and “misleading claims”. A post’s “propensity for harm” determines whether it is flagged with a label or a warning, or is removed entirely.

But the platform has not explicitly defined “misinformation” for users who will engage in the trial. So how will they know whether something is indeed “misinformation”? And what will stop users from flagging content they simply disagree with?

Familiar information feels right

As individuals, what we consider to be “true” and “reliable” can be driven by subtle cognitive biases. The more you hear certain information repeated, the more familiar it will feel. In turn, this feeling of familiarity tends to be taken as a sign of truth.

Even “deep thinkers” aren’t immune to this cognitive bias. As such, repeated exposure to certain ideas may get in the way of our ability to detect misleading content. Even if an idea is misleading, if it’s familiar enough it may still pass the test.

In direct contrast, content that is unfamiliar or difficult to process — but highly valid — may be incorrectly flagged as misinformation.

The social dilemma

Another challenge is a social one. Repeated exposure to information can also convey a social consensus, wherein our own attitudes and behaviours are shaped by what others think.

Group identity influences what information we think is factual. We think something is more “true” when it’s associated with our own group and comes from an in-group member (as opposed to an out-group member).

Research has also shown we are inclined to look for evidence that supports our existing beliefs. This raises questions about the efficacy of Twitter’s user-led experiment. Will users who participate really be capturing false information, or simply reporting content that goes against their beliefs?

More strategically, there are social and political actors who deliberately try to downplay certain views of the world. Twitter’s misinformation experiment could be abused by well-resourced and motivated identity entrepreneurs.

How to take a more balanced approach

So how can users increase their chances of effectively detecting misinformation? One way is to take a consumer-minded approach. When we make purchases as consumers, we often compare products. We should do this with information, too.

“Searching laterally”, or comparing different sources of information, helps us better discern what is true or false. This is the kind of approach a fact-checker would take, and it’s often more effective than sticking with a single source of information.

At the supermarket we often look beyond the packaging and read a product’s ingredients to make sure we buy what’s best for us. Similarly, there are many new and interesting ways to learn about disinformation tactics intended to mislead us online.

One example is Bad News, a free online game and media literacy tool which researchers found could “confer psychological resistance against common online misinformation strategies”.

There is also evidence that people who think of themselves as concerned citizens with civic duties are more likely to weigh evidence in a balanced way. In an online setting, this kind of mindset may leave people better placed to identify and flag misinformation.

Read more: Vaccine selfies may seem trivial, but they show people doing their civic duty — and probably encourage others too

Leaving the hard work to others

We know from research that thinking about accuracy or the possible presence of misinformation in a space can reduce some of our cognitive biases. So actively thinking about accuracy when engaging online is a good thing. But what happens when I know someone else is onto it?

The behavioural sciences and game theory tell us people may be less inclined to make an effort themselves if they feel like they can free-ride on the effort of others. Even armchair activism may be reduced if there is a view misinformation is being solved.

Worse still, this belief may lead people to trust information more easily. In Twitter’s case, the misinformation-flagging initiative may lead some users to think any content they come across is likely true.

Much to learn from these data

As countries engage in vaccine rollouts, misinformation poses a significant threat to public health. Beyond the pandemic, misinformation about climate change and political issues continues to present concerns for the health of our environment and our democracies.

Despite the many factors that influence how individuals identify misleading information, there is still much to be learned from how large groups come to identify what seems misleading.

Such data, if made available in some capacity, have great potential to benefit the science of misinformation. And combined with moderation and objective fact-checking approaches, it might even help the platform mitigate the spread of misinformation.