There’s a new group of AI models just launched that stands out because anyone can recreate them from the very beginning if they want to.
On Tuesday, a nonprofit organization dedicated to AI research, called Ai2 (which was founded by Paul Allen, who co-created Microsoft), introduced OLMo 2, the second version of its Open Language Model series. The term “OLMo” is short for “Open Language Model.” While there are many open-source language models out there, like Meta’s Llama, OLMo 2 meets the strict definitions of open source. This means that all the tools and data used to create it are available for anyone to access and use.
The Open Source Initiative, a well-established organization that defines and protects open-source projects, finalized its criteria for open source AI in October. The first OLMo models, which were released in February, also adhered to these guidelines.
In a blog post, AI2 explained that OLMo 2 was built from start to finish using data that is open and accessible, including the code used for training the model, detailed instructions on how to reproduce the training, clear evaluations of its performance, and even intermediary steps in the process. By sharing this information, they hope to give the open-source community the resources necessary to explore new and creative ways to use AI.
Read Also: Trump Thinks About Appointing An AI Leader
The OLMo 2 family includes two models: one with 7 billion parameters (called OLMo 7B) and another with 13 billion parameters (called OLMo 13B). In simple terms, parameters are like the skills of the model; more parameters typically mean the model can perform better at different tasks than a model with fewer parameters.
Like other language models, both OLMo 2 7B and 13B can handle a variety of text-related tasks, such as answering questions, summarizing articles, and even writing code.
To train these models, Ai2 used a massive dataset that consists of 5 trillion tokens. Tokens are snippets of data, and 1 million tokens is approximately equal to 750,000 words. The training data came from high-quality websites, academic papers, discussion boards, and a mix of both human-written and computer-generated math exercises.
Ai2 claims these new models perform very well, ranking alongside competitive open models like Meta’s Llama 3.1. They noted that there is a significant improvement in performance in OLMo 2 compared to previous versions of their models, and interestingly, the OLMo 2 7B actually outperformed the Llama 3.1 8B model.
All components of the OLMo 2 models can be downloaded from Ai2’s website, and they are licensed under Apache 2.0, which means people can use them for business purposes.
Recently, there’s been some discussion about the safety of open models, especially since some Llama models were reportedly used by researchers in China for defense projects. When asked if he was worried about potential misuse of the OLMo models, Ai2 engineer Dirk Groeneveld responded that he believes the positive impacts of sharing open models outweigh the risks. He acknowledged that while it’s possible for open models to be used inappropriately, having open access also encourages advancements in technology that can lead to more ethical AI. Furthermore, it allows for verification and reproducibility in AI research, which is only possible when the complete process is shared. This openness helps to distribute power more fairly, giving more people access to important tools and resources.
