Customizable chip for AI applications
Inventors
Naderiparizi, Saman • Rastegari, Mohammad • KHATAMIFARD, Sayyed Karen
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12475352-B2
Publication Date
2025-11-18
Expiration Date
Abstract
In one embodiment, a computing device includes an input sensor providing an input data; a programmable logic device (PLD) implementing a convolutional neural network (CNN), wherein: each compute block of the PLD corresponds to one of a multiple of convolutional layers of the CNN, each compute block of the PLD is placed in proximity to at least two memory blocks, a first one of the memory blocks serves as a buffer for the corresponding layer of the CNN, and a second one of the memory blocks stores model-specific parameters for the corresponding layer of the CNN.
Core Innovation
The invention relates to a customizable AI inference engine implemented using a programmable logic device (PLD) with a plurality of logical units. The disclosure maps convolutional neural network layers to compute blocks on the PLD, where each logical unit corresponds to a respective convolutional layer, and uses layer-specific placement of data and parameters to support per-layer execution on the PLD.
A subset of convolutional neural network layers is selected based on energy consumption when executed on at least one of a CPU or a GPU. An input is passed from the CPU or GPU to a first layer of the subset of layers to the PLD, and an output of a last layer of the subset of layers is passed from the PLD back to the CPU or GPU, with each logical unit corresponding to a respective layer of the selected subset.
The disclosure emphasizes near-memory architecture to reduce power by minimizing data movement, with a buffer memory for layer data and a model-parameter memory for layer-trained weights/parameters placed near the compute blocks. The system further supports parallel or concurrent computation of CNN layers on the PLD by arranging compute blocks and memory blocks per layer, and example embodiments include a sensor device performing object recognition with an FPGA CNN inference and an energy harvesting microsystem.
Claims Coverage
The independent claims cover a computer-implemented method, a corresponding system, and a computer program product. Across these independent claims, there are three main inventive features: selecting a subset of neural-network layers based on energy use, routing an input into the PLD for the first selected layer and returning the last selected layer’s output to a CPU/GPU, and assigning each PLD logical unit to a corresponding layer in the selected subset.
Energy-based subset layer selection between CPU/GPU and PLD
determining a subset of layers of a neural network that utilize at least a threshold amount of energy when executed on at least one of a CPU or a GPU;
Input to first selected layer on PLD and output of last selected layer back to CPU/GPU
passing an input from the at least one of the CPU or the GPU to a first layer of the subset of layers to a programmable logic device (PLD) comprising a plurality of logical units; and passing an output of a last layer of the subset of layers from the PLD to the at least one of the CPU or the GPU.
PLD logical units mapped to corresponding layers in the subset
each of the plurality of logical units corresponding to a respective layer of the subset of layers.
Taken together, the independent claims define an energy-aware split of neural-network execution between a CPU/GPU and a PLD: selected high-energy layers are executed on the PLD with logical units mapped to the corresponding layers, while the input is provided to the first selected layer and the output of the last selected layer is returned to the CPU/GPU.
Stated Advantages
Power consumption reduction for inference (up to about a factor of 100 versus CPU/GPU, as reported in the disclosure).
Lower energy per inference (about 2 mJ per inference, as reported in the disclosure).
Documented Applications
Object recognition using an FPGA CNN inference in a sensor device embodiment.
Use in an energy harvesting microsystem embodiment with an energy source including a photovoltaic cell, supercapacitor, DC-DC converter, and communication module.
Interested in licensing this patent?