GPT-4o mini is OpenAI’s latest advancement in cost-efficient artificial intelligence, designed to be more affordable and versatile.
This new iteration not only rivals but also exceeds the capabilities of previous models like GPT-3.5 Turbo, making it more accessible for a broader range of applications.
It scores an impressive 82% on the Measuring Massive Multitask Language Understanding (MMLU) test, compared to 70% for GPT-3.5 Turbo. It also offer better performance in multilingual tasks.
This model is priced at 15 cents per million input tokens and 60 cents per million output tokens, making it significantly cheaper than older models.
Its lower cost doesn’t compromise on performance, as it excels in many tasks, from handling large volumes of context to providing real-time customer support.
If you are a developer with applications running on GPT 3.5, then switching to GPT-4 o, will significantly bring down your costs.
Now, GPT-4o mini is not only affordable but also versatile, supporting both text and vision inputs.
The model is available on Azure AI and promises enhanced capability for complex tasks, with a larger context window of 128K tokens.
A Small model with superior textual intelligence and multimodal reasoning
GPT-4o mini excels in both textual intelligence and multimodal reasoning. It performs better than GPT-3.5 Turbo and other small models on key academic benchmarks.
Reasoning tasks: This model shines in reasoning tasks involving both text and vision. It scored an impressive 82.0% on MMLU, a benchmark for textual intelligence and reasoning.
This is higher than the scores of 77.9% for Gemini Flash and 73.8% for Claude Haiku.
Math and coding proficiency: GPT-4o mini also excels in math and coding tasks. On the MGSM benchmark, which measures math reasoning, it scored 87.0%. By comparison, Gemini Flash scored 75.5% and Claude Haiku scored 71.7%.
In coding tasks, measured by the HumanEval benchmark, GPT-4o mini achieved 87.2%, outperforming Gemini Flash and Claude Haiku, which scored 71.5% and 75.9%, respectively.
Multimodal reasoning: This model demonstrates strong performance on the MMMU benchmark, which tests multimodal reasoning. GPT-4o mini scored 59.4%, while Gemini Flash scored 56.1% and Claude Haiku scored 50.2%.
Besides academic benchmarks, GPT-4o mini also excels in practical applications. It performs well in function calling, enabling developers to create applications that fetch data or interact with external systems smoothly.
It also offers improved long-context performance, which allows the model to handle larger inputs more effectively.
Built-in safety measures
The safety protocols for GPT-4o mini are as robust as those for GPT-4o. The development team conducted thorough assessments using both automated tools and human evaluators based on OpenAI’s Preparedness Framework.
Over 70 experts in fields like social psychology and misinformation scrutinized GPT-4o to identify potential risks. The insights from these evaluations have been crucial in enhancing the safety of both GPT-4o and GPT-4o mini.
OpenAI continually refines the safety features of GPT-4o mini by integrating new methods developed from ongoing research. In particular, GPT-4o mini is the first to use their instruction hierarchy method in the API.
This technique strengthens the model’s resistance to various security threats such as jailbreaks, prompt injections, and system prompt extractions. This makes GPT-4o mini more reliable and safer to use in large-scale applications.
Availability and pricing
GPT-4o mini is accessible via the Assistants API, Chat Completions API, and Batch API for text and vision tasks.
Cost:
- Input tokens: 15 cents per 1 million
- Output tokens: 60 cents per 1 million (equivalent to around 2,500 pages in a standard book)
Fine-tuning support is expected to be available soon.
User access:
- Free, Plus, and Team users in ChatGPT can start using GPT-4o mini today.
- Enterprise users will gain access next week.
This rollout aligns with the goal of making AI benefits widely available.