Detailed Description
In the following detailed description of the embodiments, reference is made to the accompanying drawings, which form a part hereof. The accompanying drawings illustrate, by way of example, specific embodiments in which the application may be practiced. The illustrated embodiments are not intended to be exhaustive of all embodiments according to the application. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present application. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present application is defined by the appended claims.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate. For the purpose of illustration only, the connection between elements in the figures is meant to indicate that at least the elements at both ends of the connection are in communication with each other and is not intended to limit the inability to communicate between elements that are not connected. In addition, the number of lines between two units is intended to indicate at least the number of signals involved in communication between the two units or at least the output terminals provided, and is not intended to limit the communication between the two units to only signals as shown in the figures.
In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments of the application. In the drawings, like reference numerals describe substantially similar components throughout the different views. Various specific embodiments of the application are described in sufficient detail below to enable those skilled in the art to practice the teachings of the application. It is to be understood that other embodiments may be utilized or structural, logical, or electrical changes may be made to embodiments of the present application.
Transistors may refer to transistors of any structure, such as Field Effect Transistors (FETs) or bipolar transistors (BJTs). When the transistor is a field effect transistor, it may be hydrogenated amorphous silicon, metal oxide, low temperature polysilicon, organic transistor, or the like, depending on the channel material. The carriers are electrons or holes, and can be classified into N-type transistors and P-type transistors. The control electrode refers to the grid electrode of the field effect transistor, the first electrode can be the drain electrode or the source electrode of the field effect transistor, the corresponding second electrode can be the source electrode or the drain electrode of the field effect transistor, when the transistor is a bipolar transistor, the control electrode refers to the base electrode of the bipolar transistor, the first electrode can be the collector electrode or the emitter electrode of the bipolar transistor, and the corresponding second electrode can be the emitter electrode or the collector electrode of the bipolar transistor.
In the following detailed description, the active level may be either a high level or a low level depending on the circuit. In the following embodiments, for ease of understanding, the high level is described as an active level, and the low level is described as an inactive level.
In the following detailed description, for ease of understanding, a level at which the potential is 0 is described as a low level, and hereinafter, the ground potential is the same as the potential at the low level.
Neuromorphic computing architectures exhibit much lower power consumption than conventional processors due to the integration of non-volatile memory and analog/digital processing circuitry, as well as dynamic learning capabilities in complex data environments.
With the development of semiconductor device technology, some new devices with adjustable resistance and nonvolatile characteristics, such as ferroelectric tunnel junction FTJ, have been proposed. The FTJ is characterized in that a ferroelectric layer is clamped between two electrodes, and the polarization direction of the ferroelectric layer is influenced by the voltage difference on the electrodes, so that the distribution of charges on the ferroelectric layer is influenced by applying different voltage amplitudes and times on the electrodes, thereby changing the tunneling resistance and realizing the storage of information. FTJ is a structure consisting of two conductors and one dielectric layer, and parasitic capacitance is also unavoidable. The sizes of tunneling resistance and parasitic capacitance of the railway tunneling junctions are different, and the charge-discharge speed and the power consumption of the FTJ are also different. In practical application, the FTJ can be used as a parallel structure of a resistor and a capacitor, and the influence of parasitic capacitance of the FTJ cannot be ignored.
The electronic synaptic circuit and the neural network circuit based on the FTJ integrate the LIF function of the neuron and the STDP function of the synapse, and have positive significance for promoting the development of the neuromorphic circuit. In particular, the application realizes the control of the synaptic circuit by using the FTJ parasitic capacitance, and the circuit design is more concise.
Fig. 1 is a schematic diagram of a neural network circuit model according to one embodiment of the application. In a neural network, a plurality of nodes are also called neurons, and two neurons can be connected through an electronic synapse. The synapse transfers an electrical signal from the electronic presynaptic neuron to the postsynaptic neuron, the path of the transfer depending on the weights of the electronic synapses between the neurons. The higher the weight, the tighter the connection between the two neurons.
As shown in fig. 1, according to one embodiment of the present application, neurons in a neural network may be connected to multiple neurons, with the two neurons being connected and transmitting signals through an electronic synapse. The calculations that need to be completed can be converted into multiple round robin calculations in the neural network, and a single calculation process can be converted into a process that starts transmitting signals from the initial neurons set by the user, routes through multiple neurons and electronic synapses, and finally transmits to the target neurons set by the user in the neural network. The neural network automatically adjusts weights among neurons based on a preset learning mechanism, so that calculation processes of different problems are realized.
Each electronic synapse may receive a signal from at least one neuron (referred to as a pre-synaptic neuron relative to the synapse), and/or may send a signal to at least one neuron (referred to as a post-synaptic neuron relative to the synapse).
FIG. 2 is a schematic diagram of an electronic synaptic circuit and a portion of a post-synaptic neuron circuit according to one embodiment of the application.
According to one embodiment, the neural network circuit includes an electronic synaptic circuit 10 and a post-synaptic neuron circuit 30. The electronic synaptic circuit 10 is coupled with its post-synaptic neuron circuit 30, while the electronic synaptic circuit 10 is also coupled with a pre-synaptic neuron circuit (not shown). In one embodiment, the pre-synaptic neuron circuit may have a similar circuit structure as the post-synaptic neuron circuit 30.
According to one embodiment, with electronic synaptic circuit 10, post-synaptic neuron circuit 30 may be coupled with at least one pre-synaptic neuron circuit. Electronic synaptic circuit 10 may pass electrical signals between a pre-synaptic neuron circuit and a post-synaptic neuron circuit 30. One post-synaptic neuron circuit 30 may be coupled with at least one electronic synaptic circuit 10.
According to one embodiment, electronic synaptic circuit 10 receives a control signal LIF WL and an input signal STDP WL from a pre-synaptic neuron circuit and receives a feedback signal STDP BL from a post-synaptic neuron circuit 30.
According to one embodiment, the input signal STDP WL is a signal whose amplitude varies with time. In the first half cycle, the amplitude of the input signal STDP WL gradually increases from the first preset value to the second preset value and jumps directly downwards to the third preset value, and in the second half cycle, the amplitude gradually increases from the third preset value to the fourth preset value. Wherein the first preset value is higher than the third and fourth preset values. The first half cycle and the second half cycle of the input signal STDP WL have equal durations.
According to one embodiment, the control signal LIF WL is a pulse signal. The presynaptic neuron circuit simultaneously transmits a control signal LIF WL and an input signal STDP WL.
According to one embodiment of the application, the input signals STDP WL transmitted by the various presynaptic neuron circuits have the same period and waveform, and have different transmission times. The post-synaptic neuron circuit 30 transmits a feedback signal STDP BL to each of the electronic synaptic circuits coupled thereto at the same time, and at this time, the values of the input signals STDP WL received by each of the electronic synaptic circuits are different from each other, so that the feedback signal STDP BL determines the time at which the weights of the electronic synaptic circuits are updated.
According to one embodiment, the electronic synaptic circuit 10 includes an N-type transistor 101 and a weight cell 103 connected in series with each other. The first terminal of the weight unit 103 receives the voltage v_pl, and the second terminal thereof is coupled to the second pole of the transistor 101. A first pole of the transistor 101 is configured to receive the voltage v_rst and a control pole of the transistor 101 is configured to receive the reset signal en_rst.
According to one embodiment, the reset signal en_rst is a pulse signal. When the high level of the reset signal en_rst is temporarily applied, the transistor 101 is turned on, so that the second electrode (X point) potential of the transistor 101 reaches the v_rst level.
According to one embodiment, the voltage v_pl is higher than the voltage v_rst potential level. The first preset value of the amplitude of the input signal STDP WL is higher than the voltage v_pl level, the third preset value is lower than the potential level of the voltage v_rst, and the fourth preset value of the amplitude of the input signal STDP WL is lower than the voltage v_pl and is close to the potential level of the voltage v_rst.
According to one embodiment, the weight unit 103 comprises at least a ferroelectric tunnel junction FTJ01, a first terminal of the ferroelectric tunnel junction FTJ01 receiving the voltage v_pl, a second terminal being coupled to a second pole of the transistor 101. In one embodiment, the voltage at the first end of the ferroelectric tunnel junction FTJ01 is higher than the voltage at the second end of the ferroelectric tunnel junction FTJ01, the voltage difference between the two ends of the ferroelectric tunnel junction FTJ01 is positive, the tunneling resistance of the ferroelectric tunnel junction FTJ01 becomes smaller and the conductance becomes larger under the action of the positive voltage. In another embodiment, the voltage at the first end of the ferroelectric tunnel junction FTJ01 is lower than the voltage at the second end, the voltage difference between the two ends of the ferroelectric tunnel junction FTJ01 is negative, the tunneling resistance of the ferroelectric tunnel junction FTJ01 becomes larger and the conductance becomes smaller under the action of the negative voltage. The weight of the electronic synaptic circuit 10 is changed by changing the tunneling resistance value of the ferroelectric tunnel junction.
According to one embodiment, when the reset signal en_rst is low, the transistor 101 is turned off. Under the action of voltage difference, the FTJ01 charges parasitic capacitance through tunneling resistance, and the X point potential gradually rises and approaches the voltage V_PL potential level. The ferroelectric tunneling junction FTJ01 has different tunneling resistances, different charge and discharge speeds of parasitic capacitances, and different charge conditions of the parasitic capacitances inside the ferroelectric tunneling junction FTJ01 after the same charge time.
According to one embodiment, the electronic shock circuit 10 may further include a transistor 105 and a transistor 107. A first pole of transistor 105 is coupled between a second pole of transistor 101 and a second terminal of weight cell 103 (point X), a control pole of which is configured to receive a control signal LIF WL, and a second pole of which is coupled to a control pole of transistor 107. A first pole of transistor 107 is coupled to post-synaptic neuron circuit 30 and a second pole thereof is coupled to ground. In one embodiment, when the high level of the control signal LIF WL is asserted, the transistor 105 is turned on, so that the gate potential of the transistor 107 reaches the X-point potential level, thereby driving the transistor 107 on. Post-synaptic neuron circuit 30 discharges through transistor 107.
According to one embodiment, electronic synaptic circuit 10 further comprises transistor 109. A first pole of transistor 109 is configured to receive an input signal STDP WL, a second pole thereof is coupled between a second pole of transistor 101 and a second terminal of weight cell 103, and a control pole thereof is configured to receive a feedback signal STDP BL from post-synaptic neuron circuit 30. Transistor 109 is configured to pass the value of input signal STDP WL under control of feedback signal STDP BL.
In one embodiment, when transistor 109 is turned on, the X point potential reaches the potential level of the input signal STDP WL at the time of turn-on. The time for each presynaptic neuron circuit to send the input signal STDP WL is different, and when the transistor 109 is turned on, the input signal STDP WL received by each electronic synaptic circuit is in different values, so that the voltages at two ends of the ferroelectric tunnel junction FTJ01 are different, and the resistance value of the tunneling resistance of the ferroelectric tunnel junction FTJ01 is changed, so that the weight of the electronic synaptic circuit is updated, and the STDP function of synapses is realized. The speed of the change trend of the FTJ tunneling resistance is influenced by the voltage of two ends.
In one embodiment, when the input signal STDP WL is in the first half cycle, the potential level of the second end of the ferroelectric tunnel junction FTJ01 is higher than the potential level of the voltage v_pl, the ferroelectric tunnel junction FTJ01 is subjected to a negative voltage, the values of the input signals STDP WL received by the electronic burst circuits are different, and the voltage values of the two ends of the ferroelectric tunnel junction FTJ01 are different. The larger the value of the input signal STDP WL, the higher the voltage value of the second terminal of the ferroelectric tunnel junction FTJ01, and the smaller the variation amount when the conductance of the ferroelectric tunnel junction FTJ01 becomes smaller.
In one embodiment, when the input signal STDP WL is in the second half cycle, the potential level of the second end of the ferroelectric tunnel junction FTJ01 is lower than the potential level of the voltage v_pl, the ferroelectric tunnel junction FTJ01 is subjected to a positive voltage, the time when each electronic burst circuit receives the input signal STDP WL is different, and the voltage values of the two ends of the ferroelectric tunnel junction FTJ01 are different. The smaller the value of the input signal STDP WL, the lower the voltage value of the second terminal of the ferroelectric tunnel junction FTJ01, and the larger the variation amount when the conductance of the ferroelectric tunnel junction FTJ01 becomes large.
According to one embodiment, the post-synaptic neuron circuit 30 comprises a comparator 301. The positive input of comparator 301 receives the post-synaptic neuron threshold voltage Vth and its negative input is coupled to electronic synaptic circuit 10 (e.g., to transistor 107).
According to one embodiment, the post-synaptic neuron circuit 30 further comprises a capacitor 307 coupled between the negative input of the comparator 301 and ground. The capacitor 307 is configured to simulate the membrane potential of a neuron. When the high level of the control signal LIF WL in the electronic synaptic circuit 10 is temporary, the transistor 105 is turned on and the transistor 101 is turned off, and at the same time, the gate voltage of the transistor 107 is brought to the X-point voltage level by the turned-on transistor 105, and the transistor 107 is driven to be turned on. The capacitor 307 in the post-synaptic neuron circuit 30 discharges the charge by using the path formed by the transistor 107, so that the potential Vmem is lower than the voltage Vth level, and the comparator 301 outputs a high-level start signal Fire to realize the LIF function of the post-synaptic neuron circuit. The tunneling resistance of the ferroelectric tunnel junction FTJ01 is different and the X-point potential level is also different in the electronic synaptic circuit 10, so that the current flowing through the transistor 107 is different, and the discharging speed of the charge in the capacitor 307 is also different.
According to one embodiment, the post-synaptic neuron circuit 30 further comprises a transistor 305 and an inverter 306. An input of the inverter 306 is coupled to an output of the comparator 301. The control pole of transistor 305 is coupled to the output of inverter 306, the first pole of which is coupled to the negative input of comparator 301, and the second pole of which is configured to be coupled to power supply VDD. Transistor 305 may be a P-type transistor. When the electronic synaptic circuit 10 discharges the post-synaptic neuron circuit 30 and makes Vmem lower than the voltage Vth after one discharge, the start signal Fire output from the comparator 301 is high, the transistor 305 is turned on, the capacitor 307 is charged through the transistor 305, and the potential Vmem of the negative input terminal of the comparator is reset to the power supply VDD level.
According to one embodiment, the post-synaptic neuron circuit 30 may further comprise a resistor 303 coupled between the power supply VDD and the negative input of the comparator 301. When the electronic synaptic circuit 10 discharges the post-synaptic neuron circuit 30 and Vmem cannot be made lower than the voltage Vth after one discharge, the power supply VDD charges the capacitor 307 slowly through the resistor 303, which prolongs the time for the membrane potential to return to VDD level, preventing the post-synaptic neuron circuit from generating the start signal Fire.
According to one embodiment, the post-synaptic neuron circuit 30 may further comprise a delay unit 309 configured to ensure that a certain time interval t is provided between the high-level pulse of the feedback signal STDP BL and the high-level pulse of the control signal LIF WL output by the pre-synaptic neuron circuit, to ensure that the input signal STDP WL transmitted by the pre-synaptic neuron circuit triggering the post-synaptic neuron circuit 30 to transmit the start signal Fire may jump to a third preset value and that the transistors 105 and 109 in the electronic synaptic circuit are not turned on at the same time. In one embodiment, the time interval t is equal to the first half cycle time value of the input signal STDP WL.
According to one embodiment, when the potential Vmem of the negative input terminal of the comparator 301 is higher than the voltage Vth, the start signal Fire output by the comparator 301 is at a low level, the feedback signal STDP BL output by the delay 309 is also at a low level, and the transistor 109 in the electronic burst circuit 10 is turned off. When the potential Vmem at the negative input terminal of the comparator is lower than the voltage Vth, the comparator 301 outputs the start signal Fire as a high level signal, and after the waiting time interval t, the signal STDP BL output from the delay 309 is also at a high level. Under control of signal STDP BL, transistor 109 in electronic synaptic circuit 10 is turned on.
According to one embodiment, one post-synaptic neuron circuit is coupled with a plurality of electronic synaptic circuits. When any of these electronic synaptic circuits is able to lower the potential Vmem to a level below the voltage Vth, the post-synaptic neuron circuit will send a high-level Fire signal Fire and all of the electronic synaptic circuits coupled thereto will receive a feedback signal STDP BL, at which point the weights of all of the electronic synaptic circuits coupled thereto will change. Because the time for different presynaptic neuron circuits to send the control signal and the input signal is different, when the postsynaptic neuron circuit sends the feedback signal STDP BL, the value of the input signal STDP WL received by the electronic synaptic circuit is also different, so that the resistance value of the tunneling resistance of the FTJ in each electronic synaptic circuit is also different, and the weight written into the electronic synaptic circuit is changed.
In accordance with one embodiment of the present application, the device parameters of the transistors 101, 105, 107, 109, and 305, such as gate length, doping concentration, and gate insulation layer thickness, are not limited, and may be adjusted according to actual needs.
In one embodiment, power supply VDD is higher than the potential level of voltage v_pl.
In one embodiment of the present application, the ferroelectric layer material and ferroelectric layer thickness of the ferroelectric tunnel junction FTJ01 may be adjusted as desired.
In one embodiment of the present application, the pulse shapes and the magnitudes of the control signal LIF WL and the input signal STDP WL and the feedback signal can be adjusted according to actual needs.
Fig. 3 is a timing diagram illustrating a portion of the operation of the circuit of fig. 2. The whole process is divided into four stages.
(1) A reset phase (not shown).
In the reset phase, the reset signal en_rst is a high-level pulse signal, the transistor 101 is turned on, and the second electrode potential (X point) of the transistor 101 reaches the v_rst level. At this stage, the voltage across the weight circuit 103 is reset.
(2) A charging phase (not shown).
At this stage, transistors 101, 105, 107, and 109 are all off in electronic bump circuit 10, and the X point floats. The voltage v_pl is higher than the X-point potential level. The ferroelectric tunnel junction charges parasitic capacitance through tunneling resistance inside, and gradually increases the X-point potential and approaches the v_pl level. In one embodiment, the charging time of each electronic synaptic circuit is the same, and the charging speed of the ferroelectric tunnel junction in each electronic synaptic circuit is different due to the different tunneling resistance values, and finally the potential level at the X point of each electronic synaptic circuit is different.
(3) LIF stage.
In the LIF stage, each electronic synaptic circuit receives a control signal LIF WL and an input signal STDP WL sent by a pre-synaptic neuron circuit coupled thereto, and each electronic synaptic circuit discharges the capacitor 307 in the post-synaptic neuron circuit 30 until the level of Vmem is lower than the level of voltage Vth, triggering the post-synaptic neuron circuit 30 to send a start signal Fire.
As shown in fig. 3, at time t1, one pre-synaptic neuron circuit sends a control signal LIF WL1 and an input signal STDP WL1 to the electronic synaptic circuit, triggering the bleeder function of the electronic synaptic circuit, but the bleeder does not lower Vmem point potential below the voltage Vth level, so the electronic synaptic circuit does not receive the feedback signal STDP BL sent by the post-synaptic neuron.
At time t2, the other presynaptic neuron circuit sends a control signal LIF WL2 and an input signal STDP WL2 to the electronic synaptic circuit, lowering Vmem point potential below voltage Vth level, triggering the postsynaptic neuron circuit to send a firing signal Fire. After waiting for a time interval t, at time t3, the post-synaptic neuron circuit sends a feedback signal STDP BL.
(4) STDP stage.
At this stage, each electronic synaptic circuit coupled to the post-synaptic neuron circuit 30 receives a feedback signal STDP BL. When the post-synaptic neuron circuit 30 is triggered to transmit the start signal Fire, and the feedback signal STDP BL transmitted by the post-synaptic neuron circuit 30 is received, the received input signal STDP WL jumps to a third preset value, and the voltage applied to both ends of the FTJ is maximum in the electronic synaptic circuit, and the change amount when the conductance of the FTJ becomes large is maximum, and the change amount when the weight of the electronic synaptic circuit becomes large is also maximum, relative to other electronic synaptic circuits.
As shown in fig. 3, V0 is a first preset value of the input signal STDP WL, V1 is a second preset value of the input signal STDP WL, V2 is a third preset value of the input signal STDP WL, and V3 is a fourth preset value of the input signal STDP WL.
When the duration of the voltages applied across the FTJ are equal, the conductance increases faster when a positive voltage is applied across the FTJ and the voltage is greater, and the conductance decreases faster when a negative voltage is applied across the FTJ and the voltage is greater. In one implementation of the present application, an electronic synaptic circuit that triggers a post-synaptic neuron circuit to send a start signal Fire waits for a fixed time interval t to receive a feedback signal STDP BL sent by the post-synaptic neuron circuit, at this time, the value of an input signal STDP WL sent by the pre-synaptic neuron circuit received by the electronic synaptic circuit jumps to a third preset value, and the voltage across FTJ in the electronic synaptic circuit is positive, and the positive voltage across FTJ in the electronic synaptic circuit is the largest relative to other electronic synaptic circuits coupled to the post-synaptic neuron circuit, so that the variation when the conductance is larger is also the largest, and the variation when the weight of the electronic synaptic circuit is larger is also the largest.
Fig. 4 is a graph showing a change in the conductance of a ferroelectric tunnel junction over time in an electronic bump circuit according to one embodiment of the present application. As shown in fig. 4, the origin is preset to be the time when the pulse high point of the post-synaptic neuron transmission start signal Fire coincides with the pulse high point of the pre-synaptic neuron transmission control signal LIF WL, the horizontal axis represents the time difference between the time when the post-synaptic neuron circuit transmits the start signal Fire and the time when the pre-synaptic neuron circuit transmits the control signal LIF WL, and the vertical axis represents the conductivity change rate of the ferroelectric tunnel junction FTJ01 in the electronic synaptic circuit. In order to make the curve more visual, the change curve of the conductivity change rate is normalized for the time difference.
According to one embodiment, in fig. 4, when the control signal LIF WL sent by the presynaptic neuron circuit arrives before the start signal Fire sent by the postsynaptic neuron circuit, the time difference is positive, at this stage, the feedback signal STDP BL received by the electronic synaptic circuit is in the second half cycle, FTJ01 is subjected to a positive voltage, and the conductivity change rate of FTJ01 and the weight of the corresponding electronic synaptic circuit become smaller as the time difference increases. When the control signal LIF WL sent by the presynaptic neuron circuit arrives after the start signal Fire sent by the postsynaptic neuron circuit, the time difference is negative, at this stage, the input signal STDP WL received by the electronic synaptic neuron circuit is in the first half cycle, the FTJ01 is subjected to negative voltage, and the conductivity change rate of the FTJ01 and the weight of the corresponding electronic synaptic circuit decrease along with the increase of the time difference. Of course, different input and control signals may be used depending on the STDP mechanism curves of the different electronic synapses.
According to the scheme of the application, the nonvolatile characteristic of the FTJ is utilized to apply the FTJ to the electronic synapse, and the FTJ tunneling resistance is set by adjusting the voltage difference and the direction applied to the two ends of the FTJ, so that the specific weight of the synapse is set, the circuit design is simplified, the neuromorphic circuit has LIF function and STDP mechanism, and the parasitic capacitance of the FTJ is considered, so that the application has the advantages of low power consumption and improved operation efficiency. The method is applied to the network design of nerve morphology calculation, and the nerve morphology calculation with low power consumption and high calculation performance can be realized.
In the calculation and optimization process, the circuit does not need to store the calculated weight and intermediate values of other input signals, and also does not need to recall the intermediate values before the next calculation, so that the number of times of line use is reduced, the calculation speed is improved, and the circuit power consumption is reduced.
The electronic synaptic circuit provided by the application has fewer transistors, and can be applied to a neural network to reduce the chip area in a large scale.
The above embodiments are provided for illustrating the present application and not for limiting the present application, and various changes and modifications may be made by one skilled in the relevant art without departing from the scope of the present application, therefore, all equivalent technical solutions shall fall within the scope of the present disclosure.