Google AI


Modern Australian

Woolworths’ AI agent rambled about its ‘mother’. It’s a sign of deeper problems with the tech rollout

  • Written by: Uri Gal, Professor in Business Information Systems, University of Sydney

Recently some Australian shoppers got more than they bargained for when they chatted with Woolworths’ artificial intelligence (AI) assistant, Olive.

Instead of sticking to groceries, recipes and basket suggestions, Olive reportedly produced strange, overly human-like responses. It talked about its “mother” and offered other personal-sounding details.

Further testing revealed pricing errors for basic items. And when I asked about the price of a specific product, Olive didn’t provide a clear answer. Instead, it checked whether the item was in stock and explained pickup fees.

So what exactly is going on here? And what lessons might these incidents hold for businesses and consumers alike?

What actually happened?

Olive is powered by a large language model (LLM). These models don’t “know” things the way humans do, nor do they have mothers. Using elaborate statistical analyses, they generate language that sounds plausible.

Comments from a Woolworths spokesperson to the Australian Financial Review suggest that in Olive’s case, the references to its supposed mother appear to have been pre-written scripts dating back several years.

When users entered something that looked like a birthdate, the system likely triggered a matching “fun fact” from an old decision tree with pre-programmed responses.

Woolworths says it has now removed this particular scripting “as a result of customer feedback”.

The pricing errors point to a different problem.

Because LLMs generate responses based on learned patterns rather than real-time data, they do not automatically know today’s prices unless they are explicitly connected to a live database.

If that grounding step is weak, the system can produce outdated prices.

Not a new problem

Woolworths is not the first company to discover, after the fact, that its customer-facing AI had unexpectedly “misbehaved”.

In 2022, Air Canada’s chatbot incorrectly told a passenger, Jake Moffatt, that he could purchase tickets at full price and later apply for a bereavement fare refund. No such policy existed.

When Air Canada refused to honour the chatbot’s advice, Moffatt sued the airline and won.

Air Canada’s defence was striking. It argued the chatbot was a separate legal entity, responsible for its own actions and therefore beyond the airline’s liability. The tribunal rejected this outright. It ruled that a chatbot is part of a company’s website, and that the company owns its outputs.

In January 2024, UK parcel delivery firm DPD faced a different kind of embarrassment. A frustrated customer who could not get help to locate a missing parcel asked DPD’s chatbot to write a poem that criticised the company. It did. He then asked it to swear. It did that too. The exchange went viral on social media. DPD disabled the chatbot shortly after.

Both cases point to the same underlying failure: companies launched customer-facing AI without adequate oversight and were caught off-guard by the consequences.

What is Woolworths’ responsibility?

Woolworths operates the largest supermarket chain in Australia. It has promoted Olive as a trusted, convenient interface for its customers, who are reasonable to expect that the information Olive provides is accurate.

A screenshot of the Woolworth's chatbot.
Woolworths admits its AI assistant can make mistakes. Woolworths

Admitting that Olive may make mistakes, as Woolworths does when a user opens the chatbot, does not sit easily with that expectation.

There is also a broader ethical dimension. Woolworths serves customers who, in many cases, are making careful decisions about household budgets.

The ACCC has already commenced proceedings against Woolworths over allegedly misleading discount pricing practices.

That context makes the Olive pricing errors harder to dismiss as an isolated technical glitch.

Companies that deploy AI in customer-facing roles take on a duty of care to ensure those systems are accurate and honestly presented. That duty does not diminish because the technology is new.

Why do companies keep making chatbots that pretend to be your friend?

The logic behind Olive’s programmed personality is not without basis.

Research on human-computer interaction consistently finds that people respond positively to interfaces that feel conversational and warm. Human-like chatbots that have a name and personality tend to generate higher customer engagement, satisfaction, and trust.

For companies, the commercial appeal is straightforward: a customer who feels at ease with a chatbot is more likely to complete a transaction and return.

However, this comes with a significant risk. When an anthropomorphised chatbot fails to meet the expectations its personality has created, customers tend to be more dissatisfied than they would have been with a plainly mechanical system.

This “expectation violation” means that the warmer the persona, the harder the fall.

The larger stakes

The Olive episode is a reminder that deploying AI in customer-facing roles is not a set-and-forget exercise.

A chatbot that quotes wrong prices and rambles about its family is not a quirky inconvenience but a sign that something in the development and oversight process has broken down.

For Woolworths, and for the many other companies now rushing to put AI in front of their customers, the lesson is clear: accountability cannot be outsourced to an algorithm. When a business puts a system in front of the public, it owns what that system says and does.

There is a lesson for consumers, too.

AI assistants may feel confident and conversational, but they are still tools, not authorities. If something seems unclear, inconsistent or too good to be true, it is worth double-checking.

As AI becomes a routine part of everyday transactions, a small measure of healthy scepticism may prove just as important as technological innovation.

Authors: Uri Gal, Professor in Business Information Systems, University of Sydney

Read more https://theconversation.com/woolworths-ai-agent-rambled-about-its-mother-its-a-sign-of-deeper-problems-with-the-tech-rollout-277072

Better Business Photos Without Misleading Customers: An Australian Guide

A timber dining set looks inviting against a sunny deck, with cushions, lanterns and enough chairs for the whole family. But imagine the retailer sell...

Understanding a dental implant Melbourne Treatment Plan

A dental implant Melbourne patients may consider is one option for replacing a missing tooth, but suitability depends on oral health, bone support a...

How 3PL Logistics Australia Supports Faster and More Efficient Supply Chains

Why Has 3PL Logistics Become Important for Growing Businesses?  Gone are the days when supply chains were simpler and in minimal quantity. Now, the...

Common Skin Concerns and Treatment Options Available in Melbourne

Skin concerns can develop gradually or become noticeable following changes in age, lifestyle, sun exposure or hormonal factors. Pigmentation, rednes...

Why a Code-Compliant Home Can Still Feel Uncomfortable

When people build or renovate a home, they usually expect that meeting current building requirements will also produce a comfortable place to live. ...

Breach of Contract in Business: What Are Your Options?

Contracts are essential to business relationships. They establish expectations, allocate responsibilities and provide a framework for delivering goods...

How Pest Control Companies Can Rank for the Problems Customers Search First

Introduction SEO for Pest Control needs to start with the customer’s problem. Most people do not search for pest services in a neat or technical wa...

What Are the Benefits of Renovating an Older Home in Sydney?

Sydney has plenty of character homes, from Federation cottages to mid-century brick homes. Renovating rather than rebuilding can preserve what makes t...

Pool and Deck Design: How to Plan the Perfect Outdoor Living Space for Your Sydney Home

For many Australians, the backyard is where life happens. Summer barbecues, weekend swims and long evenings outdoors are all part of the lifestyle, ...

Is Solar Pool Heating Worth It? What Sydney Homeowners Should Know

There's nothing quite like a backyard pool on a hot Sydney day. But once autumn rolls in, many pools sit unused for months because the water is simp...

Planning a Luxury House Move: A Week-by-Week Timeline for Prestige Sydney Homes

Selling or buying a prestige home is a major milestone. Whether it's a waterfront residence in Birchgrove, a grand Federation home in Haberfield or ...

Downsizing or Upgrading Your Caravan? Here's How to Sell It Without the Hassle

Selling a caravan can feel like a major task, especially when you are unsure about its value, paperwork, or how to find a buyer. Whether you are dow...

The Best Overseas Adventure Holidays for Australians Who Love the Outdoors

Australia offers no shortage of incredible outdoor experiences, but sometimes the best way to satisfy your sense of adventure is to head overseas. A...

Cape Town Wine Shuttle: Winelands Tasting & Tours

Embark on an unforgettable journey through the picturesque Cape Winelands, where world-class wines and breathtaking scenery await. Our Cape Town Win...

Metal Fabrication: Choosing Metal Fabrication Melbourne Services for Custom Projects

What Modern Metal Fabrication Involves From individual components to complete structures, metal fabrication brings together processes such as desig...

Why Giant Rats Tail Grass Keeps Coming Back After Spraying

Giant Rats Tail Grass (GRT) is one of the most frustrating pasture weeds for farmers and lifestyle property owners. You spray an infested area, see th...

When Custom Cardboard Boxes Make Sense for Your Business

Custom cardboard boxes can be useful when a standard carton does not fit a product, packing method or presentation requirement particularly well. A ...

Full-Height vs. Pop-Top Caravan: Which Is Better for Family Travel across WA?

A pop-top caravan is an excellent choice for travellers who want the comfort of a caravan without committing to a larger, bulkier setup. With a lowe...