Call it the Reasoning Renaissance.
Since the release of OpenAI’s o1, the so-called inference model, the number of inference models from competing AI labs has exploded. In early November, quantitative trader-funded AI research company DeepSeek released a preview of its first inference algorithm, DeepSeek-R1. That same month, Alibaba’s Qwen team unveiled what it claimed was the first “open” challenger to o1.
So what opened the floodgates? The first step is to find new approaches to improve generative AI technologies. As my colleague Max Zeff recently reported, “brute force” techniques for scaling models are no longer as good as they used to be.
AI companies are under strong competitive pressure to maintain their current pace of innovation. According to one estimate, the global AI market could reach $196.63 billion in 2023 and be worth $1.81 trillion by 2030.
OpenAI, for example, has claimed that its inference model can “solve more difficult problems” than its predecessors and could represent a step change in the development of generative AI. But not everyone is convinced that inferential models are the best way forward.
Ameet Talwalkar, Associate Professor of Machine Learning Carnegie Mellon said the initial results of the inference model were “very impressive.” But at the same time, he said, he would question the “motives” of anyone who claims to know for sure how far inferential models will advance the industry.
“AI companies have a financial incentive to provide rosy predictions about the capabilities of future versions of their technology,” Talwalkar said. “We run the risk of focusing myopically on a single paradigm, which is why it is important that the broader AI research community stops blindly believing the hype and marketing efforts of these companies and instead focuses on concrete outcomes.”
Two drawbacks of inference models are (1) they are expensive and (2) they consume a lot of power.
For example, in OpenAI’s API, the company charges $15 per 750,000 words it analyzes and $60 per 750,000 words its model generates. This is between 3 and 4 times the cost of OpenAI’s latest “non-inferential” model, GPT-4o.
O1 can be used for free and without restrictions on ChatGPT, OpenAI’s AI-based chatbot platform. But earlier this month, OpenAI released o1 Pro Mode, a high-end o1 tier that costs a surprising $2,400 per year.
“The overall cost of (large language model) inference is definitely not going down,” Guy Van Den Broeck, a professor of computer science at UCLA, told TechCrunch.
One of the reasons why inference models are expensive is because they require a lot of computing resources to run. Unlike most AI, o1 and other inference models attempt to verify their own actions as they perform them. This will help you avoid some of the pitfalls that commonly cause problems with your model. The downside is that it takes longer to reach a solution.
OpenAI envisions future inference models that “think” continuously for hours, days, and even weeks. The company acknowledges that the cost of use will be higher, but it could be worth it – from groundbreaking batteries to new cancer drugs.
Today, the value proposition of inference models is less clear. Costa Huang, a researcher and machine learning engineer at the non-profit organization Ai2, points out that o1 is not a very reliable calculator. And a cursory search on social media shows numerous o1 pro mode errors.
“These inference models are specialized and can perform poorly in general domains,” Huang told TechCrunch. “Some limitations will be overcome faster than others.”
Van den Broeck argues that the inference model does not work well. actual Because it makes inferences, it limits the types of operations it can successfully handle. “True inference works for all problems, not just those that are likely (from the model’s training data),” he said. “This is a major challenge that still needs to be overcome.”
Given the strong market incentives to strengthen inference models, it is certain that models will get better over time. After all, OpenAI, DeepSeek, and Alibaba aren’t the only ones investing in this new line of AI research. VCs and founders from adjacent industries are coalescing around the idea of a future dominated by inferential AI.
But Talwalkar worries that larger labs will manage these improvements.
“Large labs obviously have competing reasons for maintaining secrecy, but this lack of transparency severely hampers the research community’s ability to capitalize on these ideas,” he said. “We expect (inference models) to evolve rapidly as more people work in this direction. But while some of the ideas will come from academia, given the financial incentives here, we expect most, if not all, of the models to come from large industrial labs like OpenAI.”
