Description:
- Autonomous vehicles
- Disaster management
- Wildlife monitoring
Abstract
USC researchers present a novel processing-in-pixel-in-memory paradigm. This approach enhances the pixel array with analog multi-channel, multi-bit convolution, batch normalization, and Rectified Linear Units support. By adopting a holistic algorithm-circuit design strategy, the paradigm seamlessly integrates with CMOS image sensor platforms, replacing memory-intensive layers of convolutional neural network models. Experimental results demonstrate a ~21x reduction in data transfer bandwidth and analog-to-digital conversions and energy efficiency gains of up to ~11×. These improvements do not compromise test accuracy, offering a promising solution for resource-intensive visual applications.
Benefit
- Enhanced processing efficiency
- Energy savings
- Seamless integration
- Uncompromised accuracy
- Cost-effective
Market Application
Demand is growing for energy-efficient on-device AI solutions to process large volumes of data from sources such as high-resolution cameras. Current methods involve converting analog voltages captured by sensor pixel arrays to digital format for AI processing. Recent research explores in-sensor processing and peripheral computation, aiming to leverage low-power analog/digital computing. However, streaming high-resolution images frame by frame between the camera and processing unit results in energy, bandwidth, and security challenges. A solution that overcomes these bottlenecks would significantly improve camera-based applications.
Publications
Datta, Gourav, et al. "A processing-in-pixel-in-memory paradigm for resource-constrained tinyml applications." Scientific Reports 12.1 (2022): 14396. https://doi.org/10.1038/s41598-022-17934-1
Stage of Development
- Validated by simulation
- Available for exclusive or non-exclusive licensing