Abacus AI, a startup specializing in AI-driven machine learning and LLMOps platforms, has released a new open-source large language model (LLM) called Liberated-Qwen1.5-72B. This model is unique in its ability to strictly follow system prompts, making it more suitable for real-world applications compared to other open-source LLMs.
Why is following system prompts important in LLM deployment? When enterprises use LLMs for tasks like customer-facing chatbots, it is crucial that the model stays on track and provides accurate responses. However, existing models often veer off-course, leading to unexpected answers or actions. For example, one chatbot allowed a user to make a legally binding offer of $1 for a car. To address this issue, Abacus developed Liberated-Qwen1.5-72B to enforce compliance with system prompts.
To create the model, Abacus fine-tuned Qwen1.5-72B using a new open-source dataset called SystemChat. This dataset consists of 7,000 synthetic conversations generated with Mistral-Medium and Dolphin-2.7-mixtral-8x7b. By training the model on this dataset, Abacus ensured that it follows system messages, even if it contradicts what the user is asking throughout the conversation.
Liberated-Qwen1.5-72B performs exceptionally well in production applications, such as chatbots that require human-like responses while adhering to specific programming. In tests conducted on MT-Bench, the model outperformed other open-source models on the HumanEval leaderboard. Additionally, on MMLU, which measures world knowledge and problem-solving abilities, the model achieved a high score of 77.13.
However, it is important to note that Liberated-Qwen1.5-72B is entirely uncensored, meaning it will answer any question, including sensitive topics, without holding back. Abacus advises users to implement their own alignment layer before exposing the model as a service to ensure appropriate content.
Currently available under the tongyi-qianwen license, Liberated-Qwen1.5-72B is expected to undergo further improvements. Abacus plans to refine the model’s performance for HumanEval and release more capable models in the future by combining the SystemChat dataset with datasets used to train their other models.
In conclusion, Abacus AI’s Liberated-Qwen1.5-72B offers a groundbreaking solution for developers seeking an open-source LLM that strictly follows system prompts. With its ability to provide human-like answers while complying with programming requirements, this model is set to revolutionize the field of AI-driven applications. As Abacus continues to refine and enhance its performance, the possibilities for real-world use cases are endless.
