Google AI


Modern Australian

A key science publishing platform is cracking down on AI slop

  • Written by: Vitomir Kovanovic, Professor and Associate Director of the Centre for Change and Complexity in Learning (C3L), Education Futures, Adelaide University
A key science publishing platform is cracking down on AI slop

The pre-print website arXiv has announced that researchers who put their names to papers which included errors clearly generated by artificial intelligence (AI) will face a year-long ban and ongoing restrictions.

The move is a response to a growing influx of AI-generated papers faced by scholarly journals as well as sites such as arXiv, which serve as unofficial platforms for research publication ahead of peer review.

However, not everyone agrees that arXiv’s response to the problem is appropriate – and the solution to the flood of AI slop research may involve more AI, not less.

The rise of bot-assisted writing

AI-generated text is on the rise everywhere. A study released last week suggests half of new articles published online are now “primarily AI-generated”.

Science is not immune to this trend. Last month, the journal Organization Science published a study of how the rise of AI has affected submissions and peer reviews since the release of ChatGPT in 2022. Reporting a dramatic rise in submitted papers and a drop in quality, the authors conclude that “the current state of AI tools, amplified by existing publish-or-perish incentives, appears to be pushing the system toward an equilibrium of more rather than better research”.

A common problem in AI-generated research writing is hallucinated citations: references to other research that does not exist.

The traditional safeguard against poor quality in scholarly publishing is peer review: another expert in the subject at hand reads the research paper and interrogates the work behind it before it can be published.

However, the peer review system was already struggling before AI. Pressured researchers often have little time or incentive to do the unpaid work of peer review.

And on arXiv, which publishes preprints – articles which have most often not been peer-reviewed – even this system is not available. Last year, flooded with AI-generated submissions, the site stopped accepting certain types of article.

A study published in January (itself a preprint) estimated around 1 in 8 papers in biomedical science now contain AI-generated text.

Most researchers would agree that AI-generated text is not a problem in itself. The problem is the lower-quality work that AI can make easy to produce.

Does the punishment fit the crime?

The ArXiv announcement doesn’t come out against AI use, but rather says

If a submission contains incontrovertible evidence that the authors did not check the results of LLM generation, this means we can’t trust anything in the paper.

This may be true as far as it goes. But the penalty – a year-long ban for all authors listed on a paper – may be out of keeping with current research practices.

In the past, research was often carried out by people working alone or in groups of two or three. In these circumstances, it seems reasonable to expect each author to take responsibility for the whole.

But research is now more collaborative than ever before. Many papers have four or five authors, and in a growing number of extreme cases papers may be credited to groups of hundreds of scientists working together, each working on their own speciality and trusting their colleagues to be doing the same.

In a case where one author of dozens or hundreds included an AI-hallucinated reference in their part of the paper, banning the lot seems harsh.

And there are no equivalent sanctions for publishing other problematic material. There’s no ban for pushing fringe or discredited theories, or using poor quality evidence and illogical arguments, for example.

Can AI help fight slop?

The rise of AI produces problems for publishers and quality assurance. And the idea of some kind of sanctions for reckless use of AI, such as included hallucinated references, is a good one.

But ArXiv’s particular choice seems drastic. If the goal is to improve peer review and quality assurance, AI systems themselves can play a role.

Modern AI systems are quite capable of taking a list of references and checking everything on it is a real paper available on the internet. Any references flagged as suspect can then be checked by a human.

AI can even be useful for carrying out quick sense-checks of things like a paper’s statistical analysis.

Perhaps this is the way forward, rather than harsh sanctions for relatively minor AI-related infractions.

Authors: Vitomir Kovanovic, Professor and Associate Director of the Centre for Change and Complexity in Learning (C3L), Education Futures, Adelaide University

Read more https://theconversation.com/a-key-science-publishing-platform-is-cracking-down-on-ai-slop-283136

The Fire Safety Checklist Every Australian Short-Stay Host Should Run

Picture an Airbnb guest arriving late, tired, in a place they’ve never been. They don’t know where the exits are, which door leads to the garage, or h...

Better Business Photos Without Misleading Customers: An Australian Guide

A timber dining set looks inviting against a sunny deck, with cushions, lanterns and enough chairs for the whole family. But imagine the retailer sell...

Understanding a dental implant Melbourne Treatment Plan

A dental implant Melbourne patients may consider is one option for replacing a missing tooth, but suitability depends on oral health, bone support a...

How 3PL Logistics Australia Supports Faster and More Efficient Supply Chains

Why Has 3PL Logistics Become Important for Growing Businesses?  Gone are the days when supply chains were simpler and in minimal quantity. Now, the...

Common Skin Concerns and Treatment Options Available in Melbourne

Skin concerns can develop gradually or become noticeable following changes in age, lifestyle, sun exposure or hormonal factors. Pigmentation, rednes...

Why a Code-Compliant Home Can Still Feel Uncomfortable

When people build or renovate a home, they usually expect that meeting current building requirements will also produce a comfortable place to live. ...

Breach of Contract in Business: What Are Your Options?

Contracts are essential to business relationships. They establish expectations, allocate responsibilities and provide a framework for delivering goods...

How Pest Control Companies Can Rank for the Problems Customers Search First

Introduction SEO for Pest Control needs to start with the customer’s problem. Most people do not search for pest services in a neat or technical wa...

What Are the Benefits of Renovating an Older Home in Sydney?

Sydney has plenty of character homes, from Federation cottages to mid-century brick homes. Renovating rather than rebuilding can preserve what makes t...

Pool and Deck Design: How to Plan the Perfect Outdoor Living Space for Your Sydney Home

For many Australians, the backyard is where life happens. Summer barbecues, weekend swims and long evenings outdoors are all part of the lifestyle, ...

Is Solar Pool Heating Worth It? What Sydney Homeowners Should Know

There's nothing quite like a backyard pool on a hot Sydney day. But once autumn rolls in, many pools sit unused for months because the water is simp...

Planning a Luxury House Move: A Week-by-Week Timeline for Prestige Sydney Homes

Selling or buying a prestige home is a major milestone. Whether it's a waterfront residence in Birchgrove, a grand Federation home in Haberfield or ...

Downsizing or Upgrading Your Caravan? Here's How to Sell It Without the Hassle

Selling a caravan can feel like a major task, especially when you are unsure about its value, paperwork, or how to find a buyer. Whether you are dow...

The Best Overseas Adventure Holidays for Australians Who Love the Outdoors

Australia offers no shortage of incredible outdoor experiences, but sometimes the best way to satisfy your sense of adventure is to head overseas. A...

Cape Town Wine Shuttle: Winelands Tasting & Tours

Embark on an unforgettable journey through the picturesque Cape Winelands, where world-class wines and breathtaking scenery await. Our Cape Town Win...

Metal Fabrication: Choosing Metal Fabrication Melbourne Services for Custom Projects

What Modern Metal Fabrication Involves From individual components to complete structures, metal fabrication brings together processes such as desig...

Why Giant Rats Tail Grass Keeps Coming Back After Spraying

Giant Rats Tail Grass (GRT) is one of the most frustrating pasture weeds for farmers and lifestyle property owners. You spray an infested area, see th...

When Custom Cardboard Boxes Make Sense for Your Business

Custom cardboard boxes can be useful when a standard carton does not fit a product, packing method or presentation requirement particularly well. A ...