Artificial Intelligence models have traditionally required massive amounts of computing power, resulting in significant costs and a large carbon footprint. In response to this, companies like Microsoft and Google have been developing smaller, lightweight models that can handle common tasks. Microsoft recently released Phi-3 Mini, a new version of its AI model designed specifically for these tasks. With 3.8 billion parameters, Phi-3 Mini is significantly smaller than other models like OpenAI’s GPT-4, making it suitable for deployment on smartphones.
One of the main advantages of lightweight AI models is their sustainability. By reducing the computing power needed to host these models, companies can make AI more accessible and environmentally friendly. This is particularly important as the industry increasingly focuses on smartphone usage. Samsung, Google, and even Apple are incorporating generative AI features into their devices, highlighting the growing demand for AI capabilities on mobile devices.
The number of parameters in an AI model determines its ability to handle complexity. While larger models with more parameters are better suited for complex requests, smaller lightweight models like Phi-3 Mini are sufficient for everyday tasks that average users require. These tasks include translating, drafting emails, and finding local restaurants. By focusing on these common tasks, Phi-3 Mini can efficiently meet user needs without the unnecessary computational requirements of larger models.
In terms of performance, Phi-3 Mini has shown promising results. It performed similarly to Meta’s open-source model Llama 3 and OpenAI’s GPT-3.5 on common benchmarks, with a few exceptions. Phi-3 Mini surpassed Llama 3 and scored just below GPT-3.5 in natural language understanding and commonsense reasoning. It also outperformed both models in arithmetic reasoning. While it scored lower on trivia and factual knowledge, researchers believe this weakness can be resolved by augmenting the model with a search engine. This means that once connected to the internet, Phi-3 Mini can overcome its limitations in accessing factual information.
To train Phi-3 Mini, researchers used a combination of heavily filtered web data and synthetic data. This challenges the belief that scraping everything from the web is the best way to train a model. Instead, they opted for curated data that meets standards for high-quality educational information. Interestingly, the model was also trained on bedtime stories, which aligns with the understanding of human brain functioning. By focusing on quality over quantity in the training data, Phi-3 Mini can achieve powerful results while running on fewer parameters.
Phi-3 Mini is now available on platforms like HuggingFace, Azure, and Ollama, providing users with access to its lightweight AI capabilities. With its smaller size and efficient performance, Phi-3 Mini represents a step forward in making AI more sustainable and accessible for everyday tasks on smartphones. As technology continues to evolve, these advancements in AI models are crucial in meeting user expectations and environmental concerns.