If you’ve ever used ChatGPT Search or Perplexity, you might have noticed that these AI tools become much more effective when they can search the internet and provide references directly in their responses. This ability to access up-to-date information helps improve the quality of answers and reduces errors, which are often referred to as “hallucinations”—a term used when an AI generates incorrect or misleading information.
To enhance this capability, a French startup called Linkup is developing a tool, known as an API, that allows developers to pull information from reliable and premium sources on the web. This information can then be used to improve the answers generated by large language models (LLMs), which are the underlying technology behind many AI chatbots. This process of combining web content with AI responses is called Retrieval-Augmented Generation, or RAG for short.
However, the future of web scraping—where bots automatically collect information from websites—is becoming more complicated. If there isn’t a financial agreement in place between the content creators and the companies that use scraping tools, it means that these bots are taking content without compensating the original publishers. This situation has led to increasing scrutiny and potential regulations regarding how AI systems are trained using online content.
Read Also: Discover How To Use Apple Intelligence’s ‘Image Playground’
There are also significant legal battles underway, such as the ongoing lawsuit between OpenAI (the company behind ChatGPT) and The New York Times, which could change the landscape for web scraping in the near future. In response to these challenges, OpenAI has started signing long-term agreements with major publishers like the Associated Press, Condé Nast, and others, ensuring that they pay for the content used in training their AI models.
Philippe Mizrahi, the co-founder and CEO of Linkup, explained that their company was founded around the same time OpenAI began making these deals with news organizations. They recognized an opportunity to help AI companies access quality content legally and ethically, benefiting both the developers and the content providers.
Currently, content publishers face tough choices about how to handle the demand for their data from AI technologies. They can block web scrapers using a file called robots.txt, which indicates whether their site can be used by AI for training. They can also take legal action against companies they believe are infringing on their copyright. Alternatively, they could allow bots to access their content freely, or they might choose to license their content to AI developers to receive compensation for their intellectual property.
However, many smaller content publishers lack the resources to pursue legal action against large tech companies. This makes it challenging for them to shift from a model that relies on scraping to one that involves licensing agreements for millions of websites.
That’s where Linkup comes in. It acts as a marketplace that connects content publishers with companies looking to enhance their AI models with web content. Linkup negotiates licensing agreements with publishers and integrates with their content management systems (CMS) so that it can access their material without scraping. In return, Linkup compensates publishers based on how often their content is used by Linkup’s clients.
Mizrahi mentioned that Linkup primarily targets businesses that are developing their own AI applications. For example, a company might use an AI model from Mistral or OpenAI to create a sales tool that generates pitches for their sales team. By using Linkup, they can access current and relevant information about potential customers, which enhances the effectiveness of their sales strategy.
Initially, Linkup focused on corporate and business information, working with news outlets as well as knowledge databases like Statista. They are not the only company trying to bring premium content to AI models through legal licensing agreements; another notable competitor is ScalePost, which collaborates with Perplexity to streamline its licensing processes.
Recently, Linkup secured €3 million (approximately $3.2 million) in seed funding from investors, including Axeleo Capital and Seedcamp. The startup currently has around 10 employees and plans to hire an additional 10 over the next year to support its growth.
