On-device Supermarket Product Recognition

Categories: Productivity, Research | Pricing: Free | Official Website ↗

This technology enables real-time identification of packaged supermarket products on mobile devices, primarily for visually impaired users.

On-device Supermarket Product Recognition is a machine learning system developed by Google Research to help visually impaired users identify packaged foods. It runs entirely on a mobile device, providing low latency and no reliance on network connectivity. The system is integrated into the Lookout Android app, where users point their smartphone camera at a product, and the app identifies it, speaking aloud the brand name and product size. The system utilizes a neural network-based approach to generate a global descriptor (embedding) for each image, significantly reducing storage requirements compared to traditional methods like SIFT or OCR+N-Gram. This efficiency allows for a larger on-device product database, improving coverage and user experience. The architecture includes components like a frame cache, IMU-based frame selector, product detector with MnasNet backbone, MediaPipe object tracker, embedder, ScaNN index searcher, and an OCR model for additional information extraction and scoring.

Key Features

On-device product detection and recognition
Real-time processing
Low latency operation
No network connectivity required
Global descriptor (embedding) generation for images
Integration with MediaPipe object tracking
Optical Character Recognition (OCR) for additional data
IMU-based frame selection for quality control

Pros

Operates entirely on-device, ensuring privacy and speed
Does not require internet connectivity
Significantly reduces storage requirements compared to traditional methods
Provides real-time product identification
Improves accessibility for visually impaired users
Robust to poor lighting and blur in images

Cons

Limited to products included in the on-device database
Initial database creation and maintenance can be complex
Performance may vary based on device processing power
Specific to Android platform (as part of Lookout app)
Not a standalone product, but a research project integrated into an app

Use Cases

Identifying packaged foods in grocery stores
Recognizing items in home cupboards
Assisting visually impaired users with daily tasks
Developing mobile accessibility applications

Best For

Visually impaired individuals
Developers of accessibility tools
Researchers in computer vision and on-device AI

Integrations: Lookout (Android app), MediaPipe

Platforms: Android

Watch demo on YouTube ↗

View full On-device Supermarket Product Recognition profile on Tools-Radar | Browse Productivity tools | Alternatives to On-device Supermarket Product Recognition

Tools-Radar is a free directory of 10,000+ AI tools — discover, compare, and choose the right AI software for your needs. Visit tools-radar.com