Disclosure of Invention
In view of the technical problems existing in the prior art, the application provides a neural network, which comprises a plurality of neuron circuits and a plurality of electronic synaptic circuits, wherein at least one of the electronic synaptic circuits is configured to receive input and control signals from one pre-synaptic neuron circuit and receive feedback signals from the one post-synaptic neuron circuit, the electronic synaptic circuit at least comprises a switch unit, a weight calculation unit and a ferroelectric transistor, wherein the switch unit is coupled between the pre-synaptic neuron circuit and the post-synaptic neuron circuit, the connection state of the electronic synaptic circuit and the post-synaptic neuron circuit is controlled under the influence of the control signals from the pre-synaptic neuron circuit, the input unit is coupled to the pre-synaptic neuron circuit and the post-synaptic neuron circuit, the input signal is received under the control of a first feedback signal and a second feedback signal from the post-synaptic neuron circuit, the weight calculation unit is coupled between the switch unit and the ground, the weight calculation unit is coupled with the input unit, and the ferroelectric transistor is configured to update the ferroelectric signal from the ferroelectric transistor.
In particular, the post-synaptic neuron circuit comprises a comparator having a positive input coupled to a switching unit of the electronic synaptic circuit, a negative input of the comparator being configured to receive a predetermined constant signal, an output of the comparator being coupled to an input unit of the electronic synaptic circuit, a resistor coupled between the positive input of the comparator and a power supply, a capacitor coupled between the positive input of the comparator and ground, the comparator being configured to output the first and second feedback signals when the voltage drops below the predetermined constant signal when the switching unit connects the post-synaptic neuron circuit to the electronic synaptic circuit and the ferroelectric transistor is turned on.
In particular, the switching unit comprises a first transistor, the control pole of which is configured to receive the control signal, the first pole of which is coupled to the positive input of the comparator, and the second pole of which is coupled to the first pole of the ferroelectric transistor.
In particular, the input unit comprises a second transistor having a control electrode configured to receive the first feedback signal, a first electrode configured to receive the input signal, a second electrode coupled to the control electrode of the ferroelectric transistor, a third transistor of a type complementary to the second transistor, a control electrode configured to receive the first feedback signal, a first electrode coupled to ground, a second electrode coupled to the control electrode of the ferroelectric transistor, a fourth transistor of the same type as the second transistor, a control electrode configured to receive the control signal, a first electrode coupled to the second electrode of the ferroelectric transistor, and a second electrode configured to receive the second feedback signal.
The control signal and the input signal have the same period, the effective level of the input signal gradually decreases from a first preset amplitude value in the first half of the effective level pulse time, and the effective level of the input signal gradually decreases from a second preset amplitude value in the second half of the effective level pulse time, wherein the second preset amplitude value is larger than the first preset amplitude value.
In particular, the effective level amplitude of the first feedback signal is greater than the second preset amplitude.
In particular, the effective level amplitude of the second feedback signal is greater than the first preset amplitude and less than the second preset amplitude.
The application provides an electronic synaptic circuit configured to receive input and control signals from a pre-synaptic neuron circuit and receive feedback signals from a post-synaptic neuron circuit, wherein the electronic synaptic circuit comprises at least a switch unit coupled between the pre-synaptic neuron circuit and the post-synaptic neuron circuit and configured to control a connection state of the electronic synaptic circuit and the post-synaptic neuron circuit under the influence of the control signals from the pre-synaptic neuron circuit, an input unit coupled to the pre-synaptic neuron circuit and the post-synaptic neuron circuit and configured to receive input signals from the pre-synaptic neuron circuit under the control of first and second feedback signals from the post-synaptic neuron circuit, and a weight calculation unit coupled between the switch unit and ground and coupled with the input unit, the weight calculation unit comprising at least a ferroelectric transistor configured to receive the input signals from the input unit and the second feedback signals to update a channel resistance of the ferroelectric transistor.
In particular, the switching unit of the electronic synaptic circuit comprises a first transistor having a control electrode configured to receive the control signal, a first electrode coupled to the input of the post-synaptic neuron and a second electrode coupled to the first electrode of the ferroelectric transistor.
In particular, the input unit of the electronic synaptic circuit comprises a second transistor having a control electrode configured to receive the first feedback signal, a first electrode configured to receive the input signal, a second electrode coupled to the control electrode of the ferroelectric transistor, a third transistor of a type complementary to the second transistor, a control electrode configured to receive the first feedback signal, a first electrode grounded, a second electrode coupled to the control electrode of the ferroelectric transistor, a fourth transistor of the same type as the second transistor, a control electrode configured to receive the control signal, a first electrode coupled to the second electrode of the ferroelectric transistor, and a second electrode configured to receive the second feedback signal.
In particular, the control electrode of the ferroelectric transistor in the electronic synaptic circuit is configured to receive the input signal, the second electrode of the ferroelectric transistor is configured to receive the second feedback signal, the control signal and the second input signal have the same period, the effective level of the input signal is gradually reduced from a first preset amplitude in the first half of the effective level pulse time, and the effective level of the first input signal is gradually reduced from a second preset amplitude in the second half of the effective level pulse time, wherein the second preset amplitude is larger than the first preset amplitude.
In particular, the magnitude of the effective level of the first feedback signal in the electronic synaptic circuit is greater than the second predetermined magnitude.
In particular, the magnitude of the effective level of the second feedback signal in the electronic synaptic circuit is greater than the first preset magnitude and less than the second preset magnitude.
The application also provides an electronic system comprising the neural network.
The application also provides electronic equipment comprising the neural network.
By adopting the scheme of the application, on one hand, the advantages of low power consumption and high calculation performance can be obtained by simulating the neurons from the physical structure of the circuit, and on the other hand, the signal transmission of the neuron form circuit is optimized by using the STDP mechanism, so that the circuit processing time can be further shortened. The ferroelectric transistor is used in circuit design, so that the power consumption of the circuit can be reduced again through the characteristics of nonvolatile property and the like, and the processing speed is improved. The electronic synaptic circuit provided by the application has fewer transistors, and can be applied to a neural network to reduce the chip area in a large scale. The novel neural network structure provided by the scheme has positive significance for promoting the development of the neuromorphic circuit.
Detailed Description
In the following detailed description of the embodiments, reference is made to the accompanying drawings, which form a part hereof. The accompanying drawings illustrate, by way of example, specific embodiments in which the application may be practiced. The illustrated embodiments are not intended to be exhaustive of all embodiments according to the application. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present application. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present application is defined by the appended claims.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate. For the purpose of illustration only, the connection between elements in the figures is meant to indicate that at least the elements at both ends of the connection are in communication with each other and is not intended to limit the inability to communicate between elements that are not connected. In addition, the number of lines between two units is intended to indicate at least the number of signals involved in communication between the two units or at least the output terminals provided, and is not intended to limit the communication between the two units to only signals as shown in the figures.
In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments of the application. In the drawings, like reference numerals describe substantially similar components throughout the different views. Various specific embodiments of the application are described in sufficient detail below to enable those skilled in the art to practice the teachings of the application. It is to be understood that other embodiments may be utilized or structural, logical, or electrical changes may be made to embodiments of the present application.
Transistors may refer to transistors of any structure, such as Field Effect Transistors (FETs) or bipolar transistors (BJTs). When the transistor is a field effect transistor, it may be hydrogenated amorphous silicon, metal oxide, low temperature polysilicon, organic transistor, or the like, depending on the channel material. The carriers are electrons or holes, and can be classified into N-type transistors and P-type transistors. The control electrode refers to the grid electrode of the field effect transistor, the first electrode can be the drain electrode or the source electrode of the field effect transistor, the corresponding second electrode can be the source electrode or the drain electrode of the field effect transistor, when the transistor is a bipolar transistor, the control electrode refers to the base electrode of the bipolar transistor, the first electrode can be the collector electrode or the emitter electrode of the bipolar transistor, and the corresponding second electrode can be the emitter electrode or the collector electrode of the bipolar transistor.
In the following detailed description, the active level may be either a high level or a low level depending on the circuit. In the following embodiments, for ease of understanding, the high level is described as an active level, and the low level is described as an inactive level.
In the following detailed description, for ease of understanding, a level at which the potential is 0 is described as a low level, and hereinafter, the ground potential is the same as the potential at the low level.
The traditional computer architecture divides the equipment into a computing unit and a storage unit, the computing unit is required to be called for completing one-time computing to compute the data to be processed, and then the computing result is stored in the storage unit. When the traditional computer architecture faces larger data and more complex calculation processes, the calculation processes need to be split into a plurality of simple calculation, the steps are repeated for a plurality of times, a plurality of intermediate values are generated before the final calculation result is obtained and stored in a storage unit, and the intermediate values are called to finish the calculation of the final result during calculation.
The artificial neural network (hereinafter referred to as neural network) provided by the application can effectively simplify the complex calculation process, and can avoid the problems caused by calculation separation.
FIG. 1 is a schematic diagram of a neural network model according to one embodiment of the application. In a neural network, a plurality of nodes are also called neurons, and two neurons can be connected through an electronic synapse. The synapse transfers an electrical signal from the electronic presynaptic neuron to the postsynaptic neuron, the path of the transfer depending on the weights of the electronic synapses between the neurons. The higher the weight, the tighter the connection between the two neurons.
As shown in fig. 1, according to one embodiment of the present application, neurons in a neural network may be connected to multiple neurons, with the two neurons being connected and transmitting signals through an electronic synapse. The calculations that need to be completed can be converted into multiple round robin calculations in the neural network, and a single calculation process can be converted into a process that starts transmitting signals from the initial neurons set by the user, routes through multiple neurons and electronic synapses, and finally transmits to the target neurons set by the user in the neural network. The neural network automatically adjusts weights among neurons based on a preset learning mechanism, so that calculation processes of different problems are realized.
Each electronic synapse may receive a signal from at least one neuron (referred to as a pre-synaptic neuron relative to the synapse), and/or may send a signal to at least one neuron (referred to as a post-synaptic neuron relative to the synapse).
FIG. 2 is a schematic diagram showing the STDP learning mechanism of electronic synapses. Wherein the time Δt of the horizontal axis represents the time difference between the time at which the post-synaptic neuron transmits a signal and the time at which the pre-synaptic neuron transmits a signal, and the Δw of the vertical axis represents the rate of change of the weight of the electronic synapse. As can be seen from fig. 2, for such an electronic synapse, when Δt is positive, the weight change rate of the electronic synapse decreases with increasing Δt.
In order to prepare an electronic synapse with the STDP learning mechanism characteristic curve, the application provides an electronic synapse circuit. In particular, ferroelectric transistors (fefets) are included in such electronic synaptic circuits.
With the development of semiconductor device technology, some new devices with adjustable resistance and nonvolatile characteristics have been proposed, including resistive random access memories, phase change memories, ferroelectric transistors (fefets), and the like. Among them, fefets have advantages of low power consumption and high operation speed, and thus have received extensive attention from the scientific research and industry.
FeFET is a novel transistor in which a layer of ferroelectric material is added between the gate electrode and the gate oxide layer of a conventional MOSFET, and the magnitude and duration of the voltage applied to the gate electrode of the FeFET can be changed by modulating the magnitude of the amount of charge induced in the gate oxide layer, so that the threshold voltage and channel resistance of the FeFET can be changed. When a specific voltage is applied to the gate source of the FeFET, the electric dipole formed in the crystal structure of the ferroelectric material will remain in line with the direction of the electric field, and even if the gate voltage is removed, the above-mentioned conduction characteristics will not change, and even after the electric field is removed, the electric dipole formed in the crystal structure of the ferroelectric material will remain in such a polarized state, so that the threshold voltage and channel resistance of the FeFET will remain unchanged until the gate source voltage changes, and the threshold voltage and channel resistance will be set again.
FIG. 3A is a schematic diagram of an electronic synaptic circuit and a portion of a post-synaptic neuron circuit according to one embodiment of the application. Fig. 3B is a timing diagram illustrating the operation of the circuit shown in fig. 3A.
As shown in fig. 3A, the electronic synaptic circuit 20 is coupled with its post-synaptic neuron circuit 30, while the electronic synaptic circuit 20 is also coupled with a pre-synaptic neuron circuit (not shown). According to one embodiment, the pre-synaptic neuron circuit may have a circuit structure similar to a post-synaptic neuron circuit.
According to one embodiment, each neuron circuit may transmit at least four signals, including IP_1 and CL_1, when transmitting signals, which may be provided to a subsequent neuron circuit by a subsequent synapse as an input signal and a control signal, and FB_1 and FB_2, which may be provided to a preceding synapse as a feedback signal, the preceding synapse being connected between the neuron circuit and the preceding neuron circuit. The term "front" or "rear" as used herein refers to a state where the signal transmission path is a signal transmission path which is input by a user and is relatively closer to the most-initiated neuron circuit, and a state where the signal transmission path is a signal transmission path which is relatively farther from the most-initiated neuron circuit.
For a particular electronic synaptic circuit, CL_1 provides a control signal to the presynaptic neuronal circuit that determines the state of connection of the electronic synaptic circuit to the postsynaptic neuronal circuit. Ip_1 is an input signal provided by the presynaptic neuron circuit, and determines an updated value of the electronic synaptic circuit weight. The feedback signals fb_1 and fb_2 are provided by the post-synaptic neuron circuit, determine the timing of modifying the current weight of the electronic synaptic circuit to an updated value, and the magnitude of fb_2 also determines the updated value of the electronic synaptic circuit weight.
According to one embodiment, as shown in fig. 3A, the electronic protruding circuit 20 may include at least a switching unit 201, an input unit 202, and a weight calculation unit 203.
According to one embodiment, the switching unit 201 may comprise at least an NMOS transistor M21, for example, the gate of which may be configured to receive the control signal cl_1 from the presynaptic neuronal circuit. As shown in fig. 3B, cl_1 includes a plurality of high-level long pulses, and when cl_1 is high, electronic synaptic circuit 20 is activated to conduct an electrical connection with post-synaptic neuron circuit 30.
According to one embodiment, the input unit 202 may include at least NMOS transistors M23 and M25, and PMOS transistor M24, for example.
According to one embodiment, the drain of M23 may be configured to receive the input signal ip_1 from one pre-synaptic neuron circuit, the source may be coupled to the weight calculation unit 203, and the gate may be configured to receive the feedback signal fb_1 from the post-synaptic neuron circuit.
According to one embodiment, the source of M24 may be coupled to the source of M23 and weight calculation unit 203, the drain thereof may be grounded, and the gate thereof may be configured to receive the feedback signal fb_1 from the post-synaptic neuron circuit.
According to one embodiment, the source of M25 may be configured to receive the feedback signal fb_2 from the post-synaptic neuron circuit, the drain may be coupled to the weight calculation unit 203, and the gate may be configured to receive the control signal cl_1 from the pre-synaptic neuron circuit.
According to one embodiment, the weight calculation unit 203 may include at least a FeFET transistor M22. According to one embodiment, the gate of M22 may be coupled to the source of M23 and the source of M24, the source of M22 may be coupled to the drain of M25, and the drain of M22 may be coupled to the source of M21.
As shown in fig. 3A, a neuron circuit, such as post-synaptic neuron circuit 30, may comprise at least a comparator 301, a resistor 302, and a capacitor 303. The positive input of comparator 301 may be coupled to electronic burst circuit 20 (e.g., may be coupled to the drain of transistor M21) while also being coupled to a power supply through resistor 302. The negative input of the comparator 301 is configured to receive a preset reference voltage Vth. Once the positive input voltage Vout of the comparator 301 is lower than the negative input voltage Vth, the neuron outputs an output signal Y (including fb_1 and fb_2, and cl_1 'and ip_1' supplied to its following neurons).
According to one embodiment of the present application, the post-synaptic neuron circuit 30 may further comprise a signal processing unit 304 configured to process the output signal Y to output feedback signals fb_1 and fb_2, ensure that the output feedback signals fb_2 and fb_1 are emitted simultaneously, have the same active level duration, have the magnitude of fb_1 smaller than that of fb_2, and have the magnitude of fb_2 high in a range conforming to the circuit design requirements.
According to one embodiment of the application, the device parameters such as the gate length, the doping concentration, the thickness of the gate insulating layer and the like of M21, M22, M23, M24 and M25 are not limited, and can be adjusted according to actual needs.
According to one embodiment, as shown in FIG. 3B, when Vout is above Vth, the post-synaptic neuron circuit does not output signal Y and therefore FB_1 and FB_2 are also low. In this case, the transistor M23 in the input unit 202 is turned off, the transistor M24 is turned on, and if cl_1 is high, the transistor M25 is turned on, and the drain of the transistor M25 receives the low level of fb_2, which corresponds to the gate and source of the FeFET transistor M22 being grounded. In this case, the threshold voltage and channel resistance of the FeFET transistor M22 are unchanged. According to one embodiment, M22 may be still on when the gate-source voltage is 0.
When cl_1 is high, the charge in capacitor 303 is discharged through the path of M22 in the electronic synaptic circuit. When Vout decreases to a level below Vth, the post-synaptic neuron circuit 30 transmits signals Y, fb_1 and fb_2 to jump to a high level.
According to one embodiment, a neuron circuit has a plurality of preceding electronic shock circuits coupled thereto. When any of these electronic synapse circuits is first able to drop Vout below Vth, the neuron circuit will emit signal Y, and all preceding electronic synapse circuits coupled thereto will receive high levels of fb_1 and fb_2.
However, if Vout fails to drop to a level below Vth after the release of cl_1 during a single high pulse, and the other previous electronic synapse circuits do not achieve this goal, the power supply continues to charge capacitor 303 after cl_1 is high, and the charge in capacitor 303 is released again the next time cl_1 is high. After several bleeds or integrations, it is possible to achieve the goal of Vout dropping to a level below Vth.
During the period cl_1 is high, M21 and M25 are turned on. When fb_1 and fb_2 are high, M24 is off, M23 is on, M23 provides ip_1 to the gate of M22, and M25 provides fb_2 to the source of M22, thereby changing the threshold voltage and channel resistance of M22, updating the weight of the electronic synaptic circuit. Of course, as described above, the arrival of the high levels of fb_1 and fb_2 may be caused by the electronic burst circuit itself, or may be caused by other electronic burst circuits coupled to the neuron circuit 30. But whichever electronic synaptic circuit initiates, neuron circuit 30 will emit fb_1 and fb_2, and thus the weights of all prior electronic synaptic circuits coupled thereto will change.
According to embodiments of the present application, since the time for each electronic synaptic circuit to receive the input signal ip_1 provided by the presynaptic neuron may be different, the threshold voltage and channel resistance change value of M22 in each electronic synaptic circuit may be different, the weight after each electronic synaptic circuit change may be different, or the threshold voltage and channel resistance of the FeFET transistor may be set to different values. But whichever electronic synaptic circuit initiates, neuron circuit 30 will emit fb_1 and fb_2, so that the weights of all prior electronic synaptic circuits coupled thereto may change.
When the control signal cl_1 goes high continuously or once again, the updated weight electronic synaptic circuit 20 will bleed the charge in the capacitor 303 with the updated channel resistance.
To achieve the electronic synaptic property shown in fig. 2, according to one embodiment, for the circuit shown in fig. 3A, as shown in fig. 3B, the high Vhh duration of cl_1 coincides completely with the high pulse duration of ip_1, or cl_1 is the same as ip_1, for each clock cycle. According to one embodiment, the high level maximum values of cl_1 and ip_1 may be different. According to various embodiments, the high maximum value of cl_1 may be greater than the high maximum value of ip_1, or may be less than or equal to the high maximum value of ip_1.
According to one embodiment, the amplitude of the IP_1 high level is gradually reduced over time from the first preset amplitude V1 during the first half of the high level pulse period, the slope of which ensures that the electronic synapse has substantially the same characteristic as the characteristic of the synapse in the third quadrant in FIG. 2, and the amplitude of the IP_1 high level pulse is first ramped from the low level to the second preset amplitude V2 and gradually reduced over time during the second half of the IP_1 high level pulse period, the slope of which ensures that the electronic synapse has substantially the same characteristic as the characteristic of the synapse in the first quadrant in FIG. 2. According to one embodiment V1 is smaller than V2.
The first half time and the second half time herein may be the same or different time periods, according to different embodiments.
According to one embodiment, to meet the turn-on rules of M23 and M25, the FB_1 high level pulse amplitude, vhh, should be greater than the maximum value of the amplitude of the high level pulse of IP_1, i.e., the second preset amplitude V2, so that M23 can turn on, and the FB_2 high level pulse amplitude, i.e., V3, should be greater than the first preset amplitude V1 of IP_1 and less than the minimum value of IP_1 in the latter half of the high level pulse period to ensure that an electronic synaptic property substantially the same as the synaptic property curve shown in FIG. 2 is achieved.
FIG. 4 is a graph showing the normalized conductance of an electronic synaptic circuit as a function of time according to one embodiment of the present application. As shown in fig. 4, Δt on the horizontal axis is a time difference between a time when the post-synaptic neuron emits the signal Y and a preset time (a preset time, i.e., a time when two preceding and following periods in the ip_1 high-level pulse period are joined, or a time when two preceding and following pulses in the ip_1 high-level pulse period are switched) (Δt may be calculated with respect to other time, but a resulting change curve differs from that of fig. 4, and the mechanism of the characteristic curve shown in fig. 2 cannot be implemented). The ΔW/W of the vertical axis represents the percent change in conductivity of M22 in the electronic synaptic circuit. In order to make the curve more visual, the change curve of the conductivity is normalized for Δt.
If the high levels of fb_1 and fb_2 fall in the latter half of the high level pulse period of ip_1, Δt is positive, at which stage ip_1 is positive and gradually decreases by the second preset amplitude V2, and the conductivity of M22 (and the corresponding weight change rate of the electronic synapse) becomes smaller as Δt increases (the weight change rate of the electronic synapse is positively correlated with the conductivity of M22). If the high levels of fb_1 and fb_2 fall in the first half of the high level pulse period of ip_1, Δt is negative, ip_1 is positive at this stage and gradually decreases by the first preset amplitude V1, and the conductivity of M22 and the weight change rate of the corresponding electronic synapse become smaller as Δt increases. The magnitude of the conductance variation of M22 is affected by the high level maximum amplitude of ip_1.
Of course, different input and control signals may be used depending on the STDP mechanism curves of the different electronic synapses.
In the scheme of the application, the memory characteristic and the nonvolatile characteristic of the FeFET are utilized to apply the FeFET to the electronic synapse, and the threshold voltage and the channel resistance of the FeFET are set by adjusting the voltage value applied to the grid electrode of the FeFET, so that the specific weight of the synapse is set, the circuit design can be simplified to a certain extent, and the functions of a plurality of elements can be realized. Meanwhile, due to the advantages of low power consumption and high operation speed, the power consumption of circuits and equipment can be further reduced, and the operation efficiency is improved. The method is applied to the network design of nerve morphology calculation, and the nerve morphology calculation with low power consumption and high calculation performance can be realized.
In the calculation and optimization process, the circuit does not need to store the calculated weight and intermediate values of other input signals, and also does not need to recall the intermediate values before the next calculation, so that the number of times of line use is reduced, the calculation speed is improved, and the circuit power consumption is reduced.
The electronic synaptic circuit provided by the application has fewer transistors, and can be applied to a neural network to reduce the chip area in a large scale.
The above embodiments are provided for illustrating the present application and not for limiting the present application, and various changes and modifications may be made by one skilled in the relevant art without departing from the scope of the present application, therefore, all equivalent technical solutions shall fall within the scope of the present disclosure.