Elon Musk’s xAI has recently made headlines with the release of its Grok-2 large language model (LLM) chatbot. Available through an $8 USD monthly subscription on the social network X, the Grok-2 chatbot has gained attention for its impressive capabilities. In the last three days, two developers at xAI, Lianmin Zheng and Saeed Maleki, have completely rewritten the inference code stack for both versions of Grok-2. This rewrite has resulted in significant improvements in speed and accuracy.
To achieve these improvements, the developers relied on SGLang, an open-source system for executing complex language model programs. Developed by researchers from top universities like Stanford and UC Berkeley, SGLang integrates a frontend language with a backend runtime, making it easier for developers to program language model applications. It supports various models, including Llama, Mistral, and LLaVA, and is compatible with models like OpenAI’s GPT-4.
The performance of Grok-2 and Grok-2-Mini has also been recognized in the third-party Lmsys Chatbot Arena leaderboard. Grok-2 has secured the #2 spot with an impressive Arena Score of 1293, tying with Google’s Gemini-1.5 Pro model. Grok-2-Mini has climbed to the #5 position with an Arena Score of 1268. Both models are proprietary to xAI, showcasing the company’s dedication to advancing AI technology.
Grok-2 has particularly excelled in mathematical tasks, ranking #1 in this category. It also performs strongly in other areas such as Hard Prompts, Coding, and Instruction-following. These achievements place Grok-2 ahead of OpenAI’s GPT-4o, which now ranks #4. The enhanced speed of Grok-2-Mini makes it an attractive option for users seeking high performance with lower computational overhead.
As xAI continues to refine its models, the AI landscape can expect further enhancements in both speed and accuracy. The addition of Grok-2 and Grok-2-Mini to the Chatbot Arena leaderboard highlights xAI’s commitment to pushing the boundaries of what AI can achieve. With ongoing innovation and a dedication to advancement, xAI is solidifying its position as a leader in the AI industry.
