Unbounded parallel implementation of deep neural networks
Inventors
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11256981-B1
Publication Date
2022-02-22
Expiration Date
Abstract
An integrated circuit (IC) has an input port receiving a first ordered stream of input values, a first set of functional circuits implementing a first aperture function, a second set of functional circuits implementing a second aperture function, additional sets of functional circuits following the first and the second set of functional circuits, each additional set in order receiving the ordered stream of output values of preceding sets as an ordered stream of input values, producing a final ordered stream of output values, and an output port receiving a last ordered output stream from the sets of functional circuits and enabling the output stream to be transmitted off the IC.
Core Innovation
The invention is an integrated circuit (IC) that receives a first ordered stream of input values from a source array at an input port, where the stream is processed in order without changing the stream ordering. A first set of functional circuits implementing a first aperture function receives the ordered stream, produces partial results by individual ones of the functional circuits as required input values are received, retains the partial results for periods of time, and combines the partial results at required points in time to produce a first ordered stream of output values. Additional sets of functional circuits follow in order, each receiving the ordered stream of output values of preceding sets as an ordered stream of input values, producing partial results, retaining them for periods of time, and combining them at required points in time to produce a final ordered stream of output values.
The IC implements multiple aperture function stages sequentially so that partial results are time-aligned through retaining partial results for periods of time and combining partial results at required points in time. The resulting output stream is transmitted off the IC via an output port receiving the last ordered output stream from the sets of functional circuits. The approach supports applying aperture functions to points in input-array order and producing output values associated with points outside the outer boundaries by synthesizing those output values.
In systems of connected ICs implementing a neural network, the invention divides the neural network into portions implemented by different ICs. A first IC implements a first portion by receiving an ordered stream from a source array and processing the stream through sequential aperture-function sets to produce an ordered output stream. A second IC and additional ICs implement subsequent portions by receiving the output stream from the previous IC as an input stream, with aperture-function functional circuits connected in order from an input port of the second IC. A final IC provides an output stream of the neural network, enabling the multi-IC system to transmit ordered outputs produced by the connected ICs.
Claims Coverage
The independent claims cover integrated circuits and systems of connected integrated circuits that implement neural network functionality using ordered streams of values processed through sequential sets of functional circuits implementing aperture functions, with partial-result retention and time-aligned combining. Across the independent claims, the main inventive elements are the ordered streaming dataflow, the staged aperture-function processing with retained partial results, and the transmission of an ordered output stream, including multi-IC partitioning.
Ordered stream aperture-function sets with retained partial results
An IC comprising an input port receiving a first ordered stream of input values from a source array; a first set of functional circuits implementing a first aperture function that receives the first ordered stream, produces partial results by individual ones as required input values are received, retains the partial results for periods of time, combines the partial results at required points in time to produce a first ordered stream of output values; a second set implementing a second aperture function that receives the first ordered stream of output values as a second ordered stream of input values and similarly produces, retains, and combines partial results to produce a second ordered stream of output values; additional sets following the first and the second set, each producing a final ordered stream of output values; and an output port receiving a last ordered output stream enabling the output stream to be transmitted off the IC.
Multi-IC connected system implementing a neural network with ordered streams
A system of connected integrated circuits implementing a neural network comprising a first IC implementing a first portion with an input port receiving a first ordered stream of input values and processing the ordered stream through first and second aperture-function sets and additional sets to produce a last ordered output stream enabling transmission off the IC; a second IC implementing a second portion comprising functional circuits implementing aperture functions connected in order from an input port of the second IC connected to the output port of the first IC and receiving the stream of output values produced by the first IC; additional ICs connected in order to receive the output stream of the previous IC; and a final IC providing an output stream of the DNN.
N-adjacent grouped ordered streaming with duplicate functional circuits
An IC receiving a first ordered stream of input values from a source array in sets of values from two or more adjacent input positions in each input interval; a first set of functional circuits implementing a first aperture function receiving the first ordered stream in sets of values from two or more input positions and producing partial results, retaining partial results for periods of time, combining at required points in time to produce a first ordered stream of output values, where the first set comprises duplicate functional circuits accommodating processing of the repeated sets of input values; a second set implementing a second aperture function receiving the first ordered stream of output values as a second ordered stream of input values to produce a second ordered stream of output values; additional sets producing a final ordered stream of output values; and an output port receiving a last ordered output stream enabling transmission off the IC.
Multi-IC connected system with repeated sets of adjacent input positions per interval
A system of connected integrated circuits implementing a neural network comprising a first IC implementing a first portion receiving a first ordered stream of input values from a source array in repeated sets of values from two or more adjacent input positions in each input interval; applying a first aperture function and a second aperture function and additional ordered sets to produce partial results, retaining partial results for periods of time, and combining them at required points in time to produce a final ordered stream of output values, with an output port receiving a last ordered output stream enabling transmission off the IC; a second IC implementing a second portion comprising functional circuits implementing aperture functions connected in order from an input port connected to the output port of the first IC and receiving the stream of output values produced by the first IC; additional ICs connected to the output port of the previous IC receiving the output stream of the previous IC as an input stream; and a final IC providing an output stream of the neural network.
Across the independent claims, the core coverage is directed to ordered streaming of values through sequential aperture-function functional circuit sets that produce partial results, retain them for periods of time, and combine them at required points in time to generate ordered streams of outputs. The claims additionally cover multi-IC systems where later ICs receive ordered output streams from earlier ICs as ordered input streams, with a final IC providing the neural network output, and a grouped streaming variant that processes sets of values from adjacent input positions with duplicate functional circuits.
Stated Advantages
Enables transmission off the IC of the ordered output stream produced by the sets of functional circuits.
Enables implementing a neural network using connected integrated circuits where a final IC provides an output stream of the DNN.
Produces output values associated with points outside the outer boundaries by synthesizing those output values.
Accommodates processing of repeated sets of input values by including duplicate functional circuits.
Documented Applications
Implementing a neural network and providing an output stream of the DNN using integrated circuits implementing a neural network portion and connected in order from input ports to output ports.
Processing neural network computations through sequential aperture-function sets and outputting an ordered final output stream transmitted off the IC.
Interested in licensing this patent?