Why the fears of AI model collapse may be overstated

AI is revolutionising everything from the development of drugs to the creation of design assets via the best AI image generators. But while models have been getting more and more powerful over the past two years, there's been a growing amount of concern about so-called AI model collapse. 

The fear is that we're reaching the point where generative AI models have become as good as they can and that future AI models will decline in quality due to AI cannibalisation. As we've reported before, research has shown that training AI image generators on AI images quickly leads to a deterioration in output. Something similar happens with chatbots, with training on synthetic data leading to increasingly nonsensical results. But is an AI meltdown really inevitable?

What is model collapse in AI?

Until now AI models have mainly been trained on quality input produced by humans, much of it scraped from the internet. But the use of AI has quickly become so widespread that the internet is now full of AI-generated content of dubious usefulness and quality. This has led to the fear that future AI models will inevitably end up being trained on data generated by earlier versions of themselves and will degrade in quality as a result.

An AI model's output isn't perfect, so if a model is then trained on its own mistakes, it becomes less and less accurate. Output becomes more unreliable with each version. AI is now so important in so many areas that this could have huge implications. Businesses could find that AI-based tools become less reliable, customers may become dissatisfied, and existing biases could become even worse if AI models lose their understanding of more marginalised groups or less common situations.

Is there a solution to model collapse?

The most obvious solution would be to filter out the AI-generated content, but it's becoming more difficult to detect, unless it's tagged in some way, as with Adobe's use of content credentials. And it's not cheap to filter all that data.

AI giants like OpenAI have been seeking to shore up their supply of quality data through deals with stock image sites and news agencies, but we may reach the point where sources of exclusively human-produced data come to an end. And it's likely that AI outputs will be regurgitated many times if people believe and replicate the AI answers they get from the likes of ChatGPT or Google.

It remains to be seen if ensuring the quality of data for future training is financially viable, but collaboration could help. The mutual sharing of data sources and training methodologies could help AI developers prevent the accidental use of AI-generated data, while periodic resets involving the introduction of fresh, human-generated data could help reduce the risk of model collapse. The proliferation of more, smaller AI models could also help protect against model collapse.

These will be important things to consider, but the situation might not be quite as drastic as the vision of AI meltdown being depicted. The research into AI model collapse so far is based on a hypothetical situation in which a model is trained entirely on synthetic data. In real world situations, it's likely that there will be at least some quality data in there. 

However the issue is resolved, it seems clear that AI is here to stay in the creative sector and is likely to continue to power most major new advances in creative software. Meanwhile, AI image generation continues to become more terrifyingly realistic with the emergence of Flux AI and Deep-Live-Cam.