Does OpenAI’s new o1 model live up to the hype?
OpenAI recently released its o1 models, which are AI models that pause to “think” before providing answers. This feature, known as multi-step reasoning, allows the models to break down complex problems into smaller steps and evaluate their accuracy. While there has been a lot of excitement surrounding these models, they do have some limitations.
Compared to OpenAI’s previous model, GPT-4o, the o1 models excel at reasoning and answering complex questions. However, they are roughly four times more expensive to use. Additionally, the o1 models lack the tools, multimodal capabilities, and speed that made GPT-4o impressive. OpenAI itself admits that GPT-4o is still the best option for most prompts and that o1 struggles with simpler tasks.
According to Ravid Shwartz Ziv, an NYU professor who studies AI models, the improvement offered by o1 is not very significant. While it may be better at certain problems, it does not provide an across-the-board improvement.
Therefore, it is important to use o1 only for the questions it is designed to help with: big ones. Most people are not currently using generative AI to answer these kinds of questions due to the limitations of current AI models. However, o1 is a tentative step in that direction.
The unique feature of o1 is its ability to think through big ideas. By breaking down complex problems into smaller steps, it allows users to walk backward from the desired outcome and evaluate their thinking process. This technique has been proposed by researchers for years but has only recently become practical.
However, the o1 models come at a higher price. In addition to paying for input and output tokens, users are also charged for reasoning tokens, which represent the hidden process of breaking down problems into smaller steps. OpenAI is not fully transparent about the details of this process to maintain its competitive advantage.
Despite its limitations, the o1 model can be helpful for tackling complicated tasks. For example, when planning Thanksgiving dinner, o1 can provide detailed advice on oven usage, cost-saving strategies, and family time management. It outperforms GPT-4o in providing useful and logical responses.
However, for simpler questions, o1 tends to overthink and provide excessive information. Asking about the location of cedar trees in America resulted in an 800+ word response detailing every variation of cedar tree, which may not be necessary.
It is important to temper expectations regarding the o1 models. While there has been a lot of hype surrounding them, OpenAI’s CEO has acknowledged that they are still flawed and limited. They may not represent the revolutionary step forward that GPT-4o represented for the industry.
The underlying principles used to create o1 have been around for years. Google used similar techniques in 2016 to create AlphaGo, the AI system that defeated a world champion in the game of Go. The debate in the AI world revolves around whether these models should act as decision-makers or tools to question human thinking.
Some view o1 as a tool to question and evaluate decision-making processes rather than a fully autonomous decision-maker. It can assist in assessing skills, evaluating time constraints, and challenging assumptions.
However, the question remains whether the value provided by o1 justifies its higher price tag. As AI models become cheaper, o1 is one of the first models to become more expensive. It is up to individuals and organizations to determine if the benefits outweigh the costs.
In conclusion, OpenAI’s o1 models offer unique features like multi-step reasoning, but they also have limitations compared to previous models. They excel at complex reasoning but struggle with simpler tasks. While they can be helpful for tackling big questions, their higher cost should be carefully considered. It is important to temper expectations and view o1 as a tool to enhance decision-making rather than a fully autonomous AI system.
