H2O AI, a company dedicated to democratizing AI, has recently unveiled Danube, a compact large language model (LLM) specifically designed for mobile applications. With 1.8 billion parameters, Danube is said to rival or outperform similarly sized models in various natural language tasks. This places it alongside other strong offerings from Microsoft, Stability AI, and Eleuther AI.
The timing of this release is strategic, as companies developing consumer devices are actively exploring the potential of offline generative AI. By running models locally on the device, users can enjoy quick assistance across functions without relying on cloud-based information. H2O’s CEO and co-founder, Sri Ambati, expressed excitement about the release of Danube, stating that it will be a game-changer for mobile offline applications.
Despite being newly announced, Danube is already being hailed for its versatility. H2O claims that the model can be fine-tuned to handle a wide range of natural language applications on small devices, including common sense reasoning, reading comprehension, summarization, and translation. To train the mini model, H2O collected a trillion tokens from diverse web sources and utilized techniques refined from Llama 2 and Mistral models to enhance its generation capabilities.
When tested on benchmarks, Danube performed on par or even better than most models in the 1-2 billion parameter category. For example, in the Hellaswag test that evaluates common sense natural language inference, Danube achieved an accuracy of 69.58%, just behind Stability AI’s Stable LM 2. Similarly, in the Arc benchmark for advanced question answering, it ranked third behind Microsoft Phi 1.5 and Stability AI’s Stable LM 2.
H2O has released Danube-1.8B under an Apache 2.0 license for commercial use. Teams interested in implementing the model for mobile use cases can download it from Hugging Face and perform application-specific fine-tuning. To simplify this process, H2O plans to release additional tooling in the near future. Additionally, the company has released a chat-tuned version of the model, H2O-Danube-1.8B-Chat, which can be utilized for conversational applications.
The availability of Danube and other small-sized models is expected to drive a surge in offline generative AI applications on phones and laptops. These models will greatly assist with tasks such as email summarization, typing, and image editing. Samsung has already embraced this trend with the launch of its S24 line of smartphones.
Overall, H2O’s Danube is poised to make a significant impact on the world of mobile AI applications. Its compact size, impressive performance, and open-source nature make it an attractive choice for developers and enterprises seeking to harness the power of AI on small devices. With the rise of offline generative AI, the possibilities for mobile applications are expanding rapidly, bringing convenience and efficiency to users worldwide.