The Promise and Perils of Synthetic Data
Is it possible for an AI to be trained entirely on data generated by another AI? While it might sound far-fetched, the concept has been around for a while and is gaining traction as access to real data becomes increasingly difficult.
Prominent companies like Anthropic, Meta, and OpenAI have begun using synthetic data to train or fine-tune their AI models. For example, Anthropic used synthetic data to train its model, Claude 3.5 Sonnet, and Meta fine-tuned its Llama 3.1 models with AI-generated data. OpenAI, too, is reportedly using synthetic data from its “reasoning” model, o1, for the upcoming Orion.
But why does AI need data in the first place? And can synthetic data truly replace real-world data in training AI models?
The Importance of Annotations
AI systems are statistical machines that learn from vast examples and make predictions based on patterns. These examples are often annotated, meaning they are labeled to help the AI understand and distinguish between different objects, concepts, or ideas.
Annotations act as teaching tools for models. For instance, if a model is shown numerous images of kitchens labeled as such, it will start to associate common features with the concept of a “kitchen.” This association helps it correctly identify new images of kitchens later. However, the quality of these annotations is critical — if a kitchen is labeled as a “cow,” the model will misidentify future kitchen images as cows.
This need for labeled data has created a booming market for annotation services. Dimension Market Research estimates that the market is worth $838.2 million today and will grow to $10.34 billion within the next decade. Although there aren’t exact figures on how many people work in this field, some estimates suggest that millions are involved in data labeling.
A Drying Data Well
There are several reasons why AI companies are seeking alternatives to human-generated labels. Humans can only label data so quickly, and their biases can influence the models they train. Additionally, paying people to label data is expensive, and errors or misunderstandings in labeling can impact model accuracy.
Moreover, data is becoming increasingly expensive and scarce. Companies like Shutterstock and Reddit are making significant profits from licensing their data to AI companies. At the same time, many websites are blocking AI scrapers from accessing their content. A recent study found that over 35% of the world’s top 1,000 websites block OpenAI’s web scraper, and around 25% of data from “high-quality” sources is now restricted.
If this trend continues, some experts predict that AI developers could run out of training data between 2026 and 2032. Fears of copyright infringement and the inclusion of inappropriate material in datasets have further driven AI companies to reconsider how they gather data.
Synthetic Alternatives
Synthetic data offers a potential solution to these challenges. With synthetic data, annotations and additional training examples can be generated automatically, reducing reliance on expensive and time-consuming human labor.
Synthetic data generation has already been adopted by many in the AI industry. Companies like Writer, Microsoft, Google, Nvidia, and Hugging Face have all used synthetic data in training their models. The synthetic data market is expected to grow rapidly, reaching an estimated value of $2.34 billion by 2030.
According to Luca Soldaini, a senior research scientist at the Allen Institute for AI, synthetic data can create training material that is difficult to obtain through traditional means. For example, Meta used its Llama 3 model to generate captions for video footage during the training of its video generator, Movie Gen. These captions were later refined by humans to add more detail.
Synthetic Risks
While synthetic data offers clear advantages, it is not without its challenges. Synthetic data suffers from the same “garbage in, garbage out” problem as all AI. If the data used to generate synthetic content is biased or flawed, the resulting data will have the same issues. For instance, underrepresented groups in the original dataset will remain underrepresented in the synthetic data.
A study from Rice University and Stanford found that over-reliance on synthetic data can degrade the quality and diversity of models. As models are trained on synthetic data, their performance can worsen over time, especially when they lack real-world diversity in the initial dataset.
There’s also the risk of hallucinations, where AI models generate inaccurate or nonsensical data. These hallucinations, if left unchecked, can compound and degrade model performance further. As a study published in Nature highlighted, models trained on erroneous data tend to produce even more errors in subsequent generations, leading to a feedback loop that reduces the accuracy and relevance of future models.
Conclusion
The rise of synthetic data presents both a promise and a peril for the AI industry. While synthetic data can help address the scarcity of real-world data, it comes with its own set of challenges, from bias replication to model degradation. For synthetic data to be a reliable tool, AI developers will need to find ways to blend it with real-world data and mitigate the risks of compounding errors.
