The StarCoder2 models have been developed under the open BigCode Project, a joint effort by ServiceNow and Hugging Face. They are being made available royalty-free under Open Responsible AI Licenses (OpenRAIL), ensuring equal access to the benefits of code generation AI for organizations of all sizes.
Harm de Vries, the lead of ServiceNow’s StarCoder2 development team, emphasized the power of open scientific collaboration and responsible AI practices in the creation of StarCoder2. He stated that the models improve generative AI performance, increase developer productivity, and enable organizations to reach their full business potential.
The StarCoder2 family consists of three models, each catering to different needs. The previous generation, StarCoder LLM, debuted in a 15 billion-parameter size and was trained on 80 programming languages. In contrast, the latest generation includes models in three different sizes: 3 billion, 7 billion, and 15 billion parameters. These models have been trained on 619 programming languages, with training data seven times larger than before.
The BigCode community employed new training techniques to ensure that the models can understand and generate low-resource programming languages like COBOL, mathematics, and program source code discussions. The 3 billion-parameter model was trained using ServiceNow’s Fast LLM framework, while the 7 billion-parameter model utilized Hugging Face’s nanotron framework. The largest model, with 15 billion parameters, was trained and optimized using Nvidia’s NeMo cloud-native framework and TensorRT-LLM software.
While the performance of these models in different coding scenarios is yet to be fully evaluated, the companies note that even the smallest 3 billion-parameter model matches the performance of the original 15 billion-parameter StarCoder LLM. Enterprise teams can choose any of the models based on their specific needs and further fine-tune them using their organizational data. These models can be utilized for various use cases, including application source code generation, workflow generation, text summarization, code completion, advanced code summarization, and code snippets retrieval.
The companies highlight that the broader and deeper training of the StarCoder2 models provides repository context, allowing for accurate and context-aware predictions. This accelerates development processes and saves engineers and developers time to focus on more critical tasks.
Jonathan Cohen, the vice president of applied research at Nvidia, believes that code LLMs can drive breakthroughs in efficiency and innovation in every industry. He emphasizes that the collaboration with ServiceNow and Hugging Face introduces secure and responsibly developed models, ensuring broader access to accountable generative AI for the global community.
Access to the StarCoder2 models is available under the Open RAIL-M license with royalty-free access and use. The supporting code can be found on the BigCode project’s GitHub repository. Alternatively, teams can download and use all three models from Hugging Face. The 15 billion-parameter model trained by Nvidia will also be available on Nvidia AI Foundation, allowing developers to experiment with it directly from their browser or via an API endpoint.
While StarCoder is not the first AI-driven code generation tool, the variety of options offered by the latest generation of the project enables enterprises to leverage LLMs in application development while saving on computing resources. Other notable players in this space include OpenAI with its Codex, which powers GitHub’s co-pilot service, Amazon with its CodeWhisper tool, Replit with its small AI coding models on Hugging Face, and Codenium, which recently secured $65 million in series B funding.
Overall, the collaboration between Nvidia, Hugging Face, and ServiceNow through the StarCoder2 project represents a significant advancement in AI-driven code generation. The availability of these models in different sizes and their training on a wide range of programming languages opens up new possibilities for developers and organizations to accelerate their coding workflows while ensuring responsible AI practices.
