Amazon’s Just Walk Out (JWO) technology is transforming the retail experience by allowing customers to enter a store, select items, and leave without stopping to pay at a cashier. Recently, journalists were invited to a secretive lab at Amazon to see the latest AI-based system that powers JWO. This system uses multi-modal foundation models and transformer-based machine learning to analyze data from various sensors in stores. By generating receipts instead of text, the system improves accuracy and makes the technology easier to deploy for retailers.
The JWO technology works by combining computer vision, sensor fusion, and machine learning to track what shoppers take from or return to shelves in a store. The process begins by creating a 3D map of the physical space using an ordinary iPhone or iPad. The store is then divided into product areas called “polygons”, and custom cameras and weight sensors are installed to track shopper interactions. By analyzing data from multiple sensors and using object recognition, the system accurately predicts whether a specific item was retained by the shopper.
Previously, the system used multiple models in a chain to process different aspects of a shopping trip. However, the new AI model processes all the information in a single transformer model, improving speed, accuracy, and cost-effectiveness. The model is trained on 3D store maps and product catalogs, enabling it to adapt to store layout changes and identify misplaced items.
One interesting aspect of JWO is its use of edge computing. Amazon has built its own edge computing devices that are deployed in stores to perform the vast majority of the reasoning on-site. This reduces latency and the need for high bandwidth connections. The edge nodes are rail-mounted enclosures that likely include Amazon GPUs such as Trainium and Inferentia2, which are more affordable and accessible alternatives to Nvidia’s GPUs. Edge computing is crucial for real-world AI inference use cases like JWO, as the data is too large to stream back to cloud-hosted models.
Another development in JWO is the integration of RFID technology. Amazon is rapidly incorporating RFID into JWO, making it easier for retailers to implement the system. With RFID gates and tags on merchandise, retailers can quickly set up a JWO-like system. This minimal infrastructure requirement makes it suitable for temporary retail settings like fairgrounds and festivals.
Building a system like JWO requires significant investment in R&D. While the exact cost of JWO’s development is unknown, it is estimated to be in the range of $250 million to $800 million. This highlights the high-risk nature of R&D in enterprise AI and complex technology integration. Large tech companies like Amazon can leverage platform effects and create economies of scale, making it more feasible for them to invest in infrastructure and R&D. For other retailers, pre-integrated, immediately deployable systems like JWO are a more practical choice.
The advances in JWO AI models demonstrate the impact of transformer architecture in AI. This breakthrough in machine learning is revolutionizing natural language processing and enabling complex tasks in frictionless retail experiences. Amazon is strategically tapping into third-party retailers as a potential source of revenue growth by offering JWO through Amazon Web Services (AWS) as a service. The integration of RFID technology into JWO expands its potential market and could lead to widespread adoption.
As AI and edge computing continue to evolve, Amazon’s JWO technology serves as an example of how hyperscale cloud providers are shaping the future of retail and other industries. By offering complex AI solutions as easily deployable services, the success of JWO and similar business models will determine the broader adoption of AI in everyday businesses.