Nvidia, the leading AI giant, has unveiled its latest software technology called Nvidia Inference Manager (NIM) at the Nvidia GTC event. NIM aims to revolutionize the deployment of generative AI large language models (LLMs) by providing optimized inference engines, industry standard APIs, and support for AI models in containers for easy and rapid deployment.
NIM is a significant milestone for the deployment of gen AI, marking Nvidia’s next-generation strategy for inference. It is expected to have a profound impact on model developers and data platforms in the AI space. The technology has already garnered support from major software vendors such as SAP, Adobe, Cadence, CrowdStrike, Getty Images, ServiceNow, and Shutterstock. Additionally, data platform vendors like BOX, Cohesity, Cloudera, Databricks, Datastax, Dropbox, NetApp, and Snowflake have also partnered with Nvidia to support NIM.
Part of the NVIDIA Enterprise AI software suite, NIM is included in the 5.0 release announced at GTC. The software package provides developers with a runtime environment to build on top of, allowing them to focus on enterprise applications. Manuvir Das, VP enterprise computing at Nvidia, emphasized during a press briefing that NIM is the best software package available for developers.
So, what exactly is Nvidia NIM? At its core, NIM is a container filled with microservices. This container can house various types of models, from open to proprietary ones, and can be deployed anywhere an Nvidia GPU is available. This includes cloud environments or even laptops. NIM containers can be deployed on platforms such as Kubernetes in the cloud, Linux servers, or serverless Function-as-a-Service models. Developers can access NIM through ai.nvidia.com to begin working with it before deployment.
Nvidia wants to make it clear that NIM is not intended to replace any prior approaches to model delivery from Nvidia. Instead, it acts as a container that includes highly optimized models for Nvidia GPUs and the necessary technologies to enhance inference.
One of the primary use cases for NIMs is in support of Retrieval Augmented Generation (RAG) deployment models. Many customers have already implemented RAGs, but the challenge lies in taking these prototypes to production and delivering real business value. NIMs, along with vector database capabilities, can help solve this problem. Leading vector database vendors like Apache Lucene, Datastax, Faiss, Kinetica, Milvus, Redis, and Weaviate are already supporting NIMs.
To further enhance RAG capabilities, Nvidia has integrated its NeMo Retriever microservices into NIM deployments. NeMo Retriever is an optimized approach for data retrieval that Nvidia announced in November 2023. By combining NIMs with NeMo Retriever, enterprises can benefit from accelerated and trained retrievers using high-quality datasets.
Nvidia’s NIM technology is set to revolutionize the deployment of AI models, making it faster, easier, and more efficient. With the support of major software vendors and data platform providers, NIM is poised to become a game-changer in the AI space. Developers can now focus on building enterprise applications on top of NIM, leaving the complexities of deployment and optimization to Nvidia. This marks a significant step forward in the field of AI and will have a profound impact on various industries seeking to leverage the power of generative AI models.