Imagine, in October 2023, based on the outcome of an internal competition, OpenAI announces its intention to ship an ‘open’ language model tasked with ‘reasoning at a level equal to or exceeding most humans.’
OpenAI has also said it will publish its first ‘open-weight’ language model since GPT-2, a change of direction for the company, which had previously restricted access to the underlying weights of the model, behind a paid subscription. The new model, which is expected to launch “in the coming months” of 2025, will have advanced reasoning capabilities.
Contents
An Open-Weight Model That Is Coming Soon
OpenAI made a surprise announcement showing its desire for an open-weight language model that embeds reasoning. This represents a strategic pivot for the company, as it has largely kept its most powerful models confidential and proprietary in recent years. It will work with developers to discuss how to make this upcoming open-weight model even more useful.
Developer feedback sessions are set to kick off in San Francisco in the weeks ahead, with further sessions around Europe and the Asia Pacific.
(Note that an open-weight language model is not the same as a fully open-source model that has been made available for users — it simply opens the trained parameters or weights — users are allowed to access or analyze or modify the model for some specific use without having access to the original training data). For those looking for a middle ground between proprietary-only solutions and open everything systems, this is your answer.
The Emergence of Reasoning in AI Models
This announcement follows the success of OpenAI’s o1 model series, which based on a radically different paradigm for language models that shifted the focus from plaintext prediction to reasoning capabilities.
Officially unveiled in September 2024, the o1 series was a project that greatly improved AI reasoning abilities. O1 models, unlike traditional language models that primarily predict the next word in a sequence, are built to “spend more time thinking through problems before they respond, much like a person would.” These models are trained with an innovative technique wherein they learn how to evolve their thinking process, focus on diverse strategies and identify their mistakes.
The results have been remarkable. The o1 model ranked in the 89th percentile at competitive programming questions (Codeforces), among the top 500 students in the US on a qualifier for the USA Math Olympiad, and surpassed human PhD-level accuracy on a benchmark of physics, biology and chemistry problems.
It uses reinforcement learning to improve both its strategy (o1) and the chain of thought it uses to make that decision. It teaches itself to see and fix its errors. It figures out how to decompose difficult steps into simpler steps. It teaches itself to take another tack when the current one isn’t effective.
Adding to the family of reasoning models
After introducing o1-preview and o1-mini, OpenAI continued to grow its reasoning model family. For developers wanting power without breaking the bank, the company has launched o3-mini, a smaller and more efficient version.
Now, an o1-pro model was recently introduced, with novel capabilities including generating multiple turns of model generations before actually producing a response. These models are available primarily through OpenAI’s API, although some versions are available to ChatGPT Plus, Team, and Enterprise users.
Models like these, which rely on reasoning and logic, have found specific application in domains with complex problem-solving needs across the industries. For telecommunications and infrastructure operators, o1’s advanced reasoning capabilities can supercharge analytics, network automation, and customer-facing applications. In health care, “ researchers apply these models to annotate cell sequencing data, and physicists use them to produce mathematical formulas representing quantum optics.”
Funding the AI Revolution
To support these ambitious developments, OpenAI intends to lock down $40 billion in a new funding round led by SoftBank Group. With this large investment, we hope to advance the state of artificial intelligence research, expand computational infrastructure and enhance AI offerings.
The company has spelt out that moving to a for-profit model is necessary in order to finalize the full investment. Allegedly OpenAI has until the end of the year to transform into a for-profit entity to capitalise fully on this funding opportunity.
New Era in Model Accessibility
OpenAI’s release of an open-weight model with reasoning capabilities comes at a time of mounting competitive autocratic pressure as its rivals have increasingly been embracing more open-minded approaches to releasing models.
The planned launch of this model suggests OpenAI taking these steps to keep on top of AI developments instead of just focusing on its proprietary models as of October 2023.
While developers will need to learn more details about this new open-weight model in the future, the announcement itself has the potential to revolutionise the sharing and development of powerful AI technology, presenting an exciting prospect for global artificial intelligence innovation.
