AI doesn’t really ‘learn’ – and knowing why will help you use it more responsibly
- Written by Kai Riemer, Professor of Information Technology and Organisation, University of Sydney
What if we told you that artificial intelligence (AI) systems such as ChatGPT don’t actually learn? Many people we talk to are genuinely surprised to hear this.
Even AI systems themselves will often tell you confidently that they are learning systems. Many reports and even academic papers say the same. But this is due to a misconception – or rather a loose understanding of what we mean by “learning” in AI.
Yet, understanding more precisely how and when AI systems learn (and when they don’t) will make you a more productive and more responsible user of AI.
AI does not learn – at least not like humans do
Many misconceptions around AI stem from using words that have a certain meaning when applied to humans, such as learning. We know how humans learn, because we do it all the time. We have experiences; we do something that fails; we encounter something new; we read something surprising; and thus we remember, we update or change the way we do things.
This is not how AI systems learn. There are two main differences.
Firstly, AI systems do not learn from any specific experiences, which would allow them to understand things the way we humans do. Rather they “learn” by encoding patterns from vast amounts data – using mathematics alone. This happens during the training process, when they are built.
Take large language models, such as GPT-4, the technology that powers ChatGPT. In a nutshell, it learns by encoding mathematical relationships between words (actually, tokens), with the aim to make predictions about what text goes with what other text. These relationships are extracted from vast amounts of data and encoded during a computationally intensive training phase.
This form of “learning” is obviously very different to how humans learn.
It has certain downsides in that AI often struggles with simple commonsense knowledge about the world that humans naturally learn by just living in the world.
But AI training is also incredibly powerful, because large language models have “seen” text at a scale far beyond what any human can comprehend. That’s why these systems are so useful with language-based tasks, such as writing, summarising, coding, or conversing. The fact these systems don’t learn like us, but at a vast scale, makes them all-rounders in the kinds of things they do excel at.
Once trained, the learning stops
Most AI systems that most people use, such as ChatGPT, also do not learn once they are built. You could say AI systems don’t learn at all – training is just how they’re built, it’s not how they work. The “P” in GPT literally stands for “pre-trained”.
In technical terms, AI systems such as ChatGPT only engage in “training-time learning”, as part of their development, not in “run-time learning”. Systems that learn as they go do exist. But they are typically confined to a single task, for example your Netflix algorithm recommending what to watch. Once it’s done, it’s done, as the saying goes.
Being “pre-trained” means large language models are always stuck in time. Any updates to their training data require highly costly retraining, or at least so-called fine-tuning for smaller adjustments.
That means ChatGPT does not learn from your prompts on an ongoing basis. And out of the box, a large language model does not remember anything. It holds in its memory only whatever occurs in a single chat session. Close the window, or start a new session, and it’s a clean sheet every time.
There are ways around this, such as storing information about the user, but they are achieved at the application level; the AI model itself does not learn and remains unchanged until retrained (more on that in a moment).
What does this mean for users?
First, be aware of what you get from your AI assistant.
Learning from text data means systems such as ChatGPT are language models, not knowledge models. While it is truly amazing how much knowledge gets encoded via the mathematical training process, these models are not always reliable when asked knowledge questions.
Their real strength is working with language. And don’t be surprised when responses contain outdated information given they are frozen in time, or that ChatGPT does not remember any facts you tell it.
The good news is AI developers have come up with some clever workarounds. For example, some versions of ChatGPT are now connected to the internet. To provide you with more timely information they might perform a web search and insert the result into your prompt before generating the response.
Another workaround is that AI systems can now remember things about you to personalise their responses. But this is done with a trick. It is not that the large language model itself learns or updates itself in real time. The information about you is stored in a separate database and is inserted into the prompt each time in ways that remain invisible.
But it still means that you can’t correct the model when it gets something wrong (or teach it a fact), which it would remember to correct its answers for other users. The model can be personalised to an extent, but it still does not learn on the fly.
Users who understand how exactly AI learns – or doesn’t – will invest more in developing effective prompting strategies, and treat the AI as an assistant – one that always needs checking.
Let the AI assist you. But make sure you do the learning, prompt by prompt.
Authors: Kai Riemer, Professor of Information Technology and Organisation, University of Sydney
