Disclosure of Invention
In view of this, the present disclosure provides at least a chip verification method, a device, an electronic apparatus, and a storage medium.
In a first aspect, the present disclosure provides a chip verification method, including:
acquiring a neural network to be checked;
Generating a target operator consistent with the operator type aiming at the operator type of at least one original operator included in the neural network to be checked;
replacing the corresponding original operator in the neural network to be verified by using the target operator to generate a replaced neural network;
training the replaced neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network;
And generating a verification result corresponding to the chip to be verified based on the first training result and the second training result.
In the method, after the neural network to be checked is obtained, a target operator consistent with the operator type of at least one original operator included in the neural network to be checked is generated, and the simulation of floating point operation in the chip to be checked by using the target operator is realized because the target data processing mode corresponding to the target operator is matched with the preset data processing mode corresponding to the operator type indicated by the design file of the chip to be checked. And then the original operator corresponding to the neural network to be checked can be replaced by the target operator, and the replaced neural network is generated. And training the neural network to be verified and the replaced neural network respectively by using the training samples to obtain a first training result and a second training result. The first training result can be determined as a reference result of the network to be checked, and the second training result can be determined as a simulation result when the chip to be checked runs the neural network to be checked. And then the first training result and the second training result are used, so that the verification result of the chip to be verified can be accurately generated, and the evaluation of the accuracy of the chip to be verified is realized.
In a possible implementation manner, the generating, for the operator type of at least one original operator included in the neural network to be verified, a target operator consistent with the operator type includes:
Generating an initial operator corresponding to at least one operator type of the original operator included in the neural network to be checked;
testing the initial operator to generate a test result corresponding to the initial operator;
The method comprises the steps of obtaining a test result by taking an initial operator after adjustment as a new initial operator, and returning to the step of testing the initial operator until the test result of the initial operator after adjustment indicates that the test is passed;
And determining the initial operator as a target operator consistent with the operator type in the condition that the test result indicates that the test is passed.
Here, for at least one operator type of the original operator, after generating an initial operator corresponding to the operator type, the initial operator may be tested. When the test fails, the initial operator can be adjusted until the test passes. The initial operator after the test is passed is determined to be the target operator consistent with the operator type, so that the performance of the generated target operator is guaranteed, and the network performance deviation between the replaced neural network and the neural network to be checked is smaller when the replaced neural network is obtained based on the target operator later. And further, based on the replaced neural network and the neural network to be verified with smaller deviation, the chip to be verified can be verified accurately.
In a possible implementation manner, the testing the initial operator, generating a test result corresponding to the initial operator, includes:
Inputting preset characteristic data into the initial operator to obtain first detection data corresponding to the initial operator, wherein the first detection data comprises predicted output data obtained when the preset characteristic data comprises input characteristic data and/or predicted gradient data obtained when the preset characteristic data comprises input gradient data, and
Inputting the preset characteristic data to an original operator matched with the original operator to obtain second detection data corresponding to the original operator;
and generating a test result corresponding to the initial operator based on the first detection data and the second detection data.
In a possible implementation manner, the training the post-replacement neural network by using the training sample to obtain a second training result corresponding to the post-replacement neural network includes:
training the replaced neural network on the selected training frame by utilizing the training sample according to a first floating point operation mode indicated by the training frame to obtain a first floating point training result and/or,
Training the replaced neural network on the selected training frame by utilizing the training sample according to a second floating point operation mode used by the chip to be tested to obtain a second floating point training result;
And determining at least one of the first floating point training result and the second floating point training result as a second training result corresponding to the replaced neural network.
According to the embodiment, the replaced neural network can be trained according to the first floating point operation method on the selected training frame, and/or the replaced neural network can be trained according to the second floating point operation method, so that the training mode is flexible, the flexibility and diversity of the second training result are improved, the first training result and the second training result are used later, and the chip to be tested can be checked more accurately.
In a possible implementation manner, the generating, based on the first training result and the second training result, a verification result corresponding to the chip under test includes:
Generating a verification result indicating that the chip to be verified passes under the condition that a first deviation value between a first floating point training result in the second training results and the first training result is smaller than or equal to a set first threshold value and a second deviation value between the first floating point training result and a second floating point training result in the second training results is smaller than or equal to a set second threshold value;
generating a verification result indicating that the design file of the chip to be verified is to be adjusted under the condition that a first deviation value between the first floating point training result and the first training result is larger than a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is smaller than or equal to a set second threshold value;
Generating a verification result indicating that a floating point number operation mode used by the chip to be verified is to be adjusted under the condition that a first deviation value between the first floating point training result and the first training result is smaller than or equal to a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value;
And under the condition that a first deviation value between the first floating point training result and the first training result is larger than a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value, generating a floating point number operation mode used for indicating the chip to be tested and a verification result to be adjusted of the design file.
Here, through setting up multiple mode, the check result of chip that waits to test is more accurate and more appearance.
In a possible implementation manner, the training, using a training sample, is to train the neural network after replacement and the neural network to be verified, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the neural network after replacement, where the training sample includes:
Initializing the replaced neural network and the neural network to be checked;
And respectively carrying out multiple times of training on the initialized replaced neural network and the neural network to be checked by utilizing the training sample until the training times are equal to the set first time threshold value, and obtaining a first training result corresponding to the neural network to be checked and a second training result corresponding to the replaced neural network.
Here, the training times of the neural network can be flexibly controlled by setting the first time threshold value, and the verification efficiency of the chip can be improved by reducing the first time threshold value.
In a possible implementation manner, the training, using a training sample, is to train the neural network after replacement and the neural network to be verified, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the neural network after replacement, where the training sample includes:
initializing the neural network to be checked;
training the initialized neural network to be checked for multiple times by using a first training sample in the training samples to obtain a neural network to be checked containing first network parameters;
And respectively carrying out multiple times of training on the neural network to be verified containing the first network parameters and the replaced neural network containing the first network parameters by using a second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.
In a possible implementation manner, the training, using a training sample, is to train the neural network after replacement and the neural network to be verified, to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the neural network after replacement, where the training sample includes:
Initializing the replaced neural network and the neural network to be checked;
Respectively training the initialized replaced neural network and the neural network to be checked for multiple times by using a first training sample in the training samples until the training times are equal to a set second time threshold value, so as to obtain the replaced neural network containing the second network parameters and the neural network to be checked containing the third network parameters;
And respectively carrying out multiple times of training on the neural network to be verified containing the second network parameters and the neural network to be verified containing the third network parameters by using a second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.
The following description of the effects of the apparatus, the electronic device, etc. refers to the description of the above method, and will not be repeated here.
In a second aspect, the present disclosure provides a chip authentication apparatus, comprising:
the acquisition module is used for acquiring the neural network to be checked;
the first generation module is used for generating a target operator consistent with the operator type aiming at the operator type of at least one original operator included in the neural network to be checked;
the second generation module is used for replacing the corresponding original operator in the neural network to be verified by using the target operator to generate a replaced neural network;
The training module is used for training the replaced neural network and the neural network to be verified by utilizing training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network;
And the third generation module is used for generating a verification result corresponding to the chip to be verified based on the first training result and the second training result.
In a possible implementation manner, the first generation module is configured to, when generating, for at least one operator type of an original operator included in the neural network to be verified, a target operator consistent with the operator type:
Generating an initial operator corresponding to at least one operator type of the original operator included in the neural network to be checked;
testing the initial operator to generate a test result corresponding to the initial operator;
The method comprises the steps of obtaining a test result by taking an initial operator after adjustment as a new initial operator, and returning to the step of testing the initial operator until the test result of the initial operator after adjustment indicates that the test is passed;
And determining the initial operator as a target operator consistent with the operator type in the condition that the test result indicates that the test is passed.
In a possible implementation manner, the first generating module is configured to, when testing an initial operator and generating a test result corresponding to the initial operator:
Inputting preset characteristic data into the initial operator to obtain first detection data corresponding to the initial operator, wherein the first detection data comprises predicted output data obtained when the preset characteristic data comprises input characteristic data and/or predicted gradient data obtained when the preset characteristic data comprises input gradient data, and
Inputting the preset characteristic data to an original operator matched with the original operator to obtain second detection data corresponding to the original operator;
and generating a test result corresponding to the initial operator based on the first detection data and the second detection data.
In a possible implementation manner, the training module is configured to, when training the post-replacement neural network by using the training sample to obtain a second training result corresponding to the post-replacement neural network:
training the replaced neural network on the selected training frame by utilizing the training sample according to a first floating point operation mode indicated by the training frame to obtain a first floating point training result and/or,
Training the replaced neural network on the selected training frame by utilizing the training sample according to a second floating point operation mode used by the chip to be tested to obtain a second floating point training result;
And determining at least one of the first floating point training result and the second floating point training result as a second training result corresponding to the replaced neural network.
In a possible implementation manner, the third generating module is configured to, when generating the verification result corresponding to the chip under test based on the first training result and the second training result:
Generating a verification result indicating that the chip to be verified passes under the condition that a first deviation value between a first floating point training result in the second training results and the first training result is smaller than or equal to a set first threshold value and a second deviation value between the first floating point training result and a second floating point training result in the second training results is smaller than or equal to a set second threshold value;
generating a verification result indicating that the design file of the chip to be verified is to be adjusted under the condition that a first deviation value between the first floating point training result and the first training result is larger than a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is smaller than or equal to a set second threshold value;
Generating a verification result indicating that a floating point number operation mode used by the chip to be verified is to be adjusted under the condition that a first deviation value between the first floating point training result and the first training result is smaller than or equal to a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value;
And under the condition that a first deviation value between the first floating point training result and the first training result is larger than a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value, generating a floating point number operation mode used for indicating the chip to be tested and a verification result to be adjusted of the design file.
In a possible implementation manner, the training module is configured to, when training the post-replacement neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the post-replacement neural network, respectively:
Initializing the replaced neural network and the neural network to be checked;
And respectively carrying out multiple times of training on the initialized replaced neural network and the neural network to be checked by utilizing the training sample until the training times are equal to the set first time threshold value, and obtaining a first training result corresponding to the neural network to be checked and a second training result corresponding to the replaced neural network.
In a possible implementation manner, the training module is configured to, when training the post-replacement neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the post-replacement neural network, respectively:
initializing the neural network to be checked;
training the initialized neural network to be checked for multiple times by using a first training sample in the training samples to obtain a neural network to be checked containing first network parameters;
And respectively carrying out multiple times of training on the neural network to be verified containing the first network parameters and the replaced neural network containing the first network parameters by using a second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.
In a possible implementation manner, the training module is configured to, when training the post-replacement neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the post-replacement neural network, respectively:
Initializing the replaced neural network and the neural network to be checked;
Respectively training the initialized replaced neural network and the neural network to be checked for multiple times by using a first training sample in the training samples until the training times are equal to a set second time threshold value, so as to obtain the replaced neural network containing the second network parameters and the neural network to be checked containing the third network parameters;
And respectively carrying out multiple times of training on the neural network to be verified containing the second network parameters and the neural network to be verified containing the third network parameters by using a second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.
In a third aspect, the present disclosure provides an electronic device comprising a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory in communication over the bus when the electronic device is in operation, the machine-readable instructions when executed by the processor performing the steps of the chip authentication method as described in the first aspect or any of the embodiments above.
In a fourth aspect, the present disclosure provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the chip authentication method according to the first aspect or any of the embodiments described above.
The foregoing objects, features and advantages of the disclosure will be more readily apparent from the following detailed description of the preferred embodiments taken in conjunction with the accompanying drawings.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only some embodiments of the present disclosure, not all embodiments. The components of the embodiments of the present disclosure, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be made by those skilled in the art based on the embodiments of this disclosure without making any inventive effort, are intended to be within the scope of this disclosure.
The deep learning is a computationally intensive algorithm, and along with the increasing diversity and complexity of processing tasks, the requirements on the accuracy and instantaneity of the algorithm are continuously increased, so that the scale of the neural network used by the deep learning is increased, and more computing resources and storage resources are needed when the neural network is operated. Thus, the neural network can be run using the designed AI chip. In general, the designed AI chip needs to be evaluated before the AI chip streams.
For example, the chip area, power consumption, calculation power, accuracy, and other indicators of the AI chip may be evaluated. Since accuracy is a main index for measuring the performance of an AI chip, it is particularly important to provide a method for evaluating the accuracy of the AI chip. Based on the above, the embodiment of the disclosure provides a chip verification method, a device, an electronic device and a storage medium.
The defects of the scheme are all results obtained by the inventor after practice and careful study, and therefore, the discovery process of the above problems and the solutions to the above problems set forth hereinafter by the present disclosure should be all contributions of the inventors to the present disclosure during the course of the present disclosure.
It should be noted that like reference numerals and letters refer to like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
For the convenience of understanding the embodiments of the present disclosure, a chip verification method disclosed in the embodiments of the present disclosure will be described in detail first. The main execution body of the chip verification method provided by the embodiment of the disclosure is generally a computer device with a certain computing capability, where the computer device includes, for example, a terminal device or a server or other processing devices, and the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a Personal digital assistant (Personal DIGITAL ASSISTANT, PDA), and so on. In some possible implementations, the chip authentication method may be implemented by way of a processor invoking computer readable instructions stored in a memory.
Referring to fig. 1, a flow chart of a chip verification method according to an embodiment of the disclosure is shown, and the method includes S101-S105, where:
s101, acquiring a neural network to be checked.
S102, generating a target operator consistent with an operator type aiming at the operator type of at least one original operator included in the neural network to be checked;
And S103, replacing the corresponding original operator in the neural network to be verified by using the target operator, and generating a replaced neural network.
And S104, training the replaced neural network and the neural network to be checked by using training samples to obtain a first training result corresponding to the neural network to be checked and a second training result corresponding to the replaced neural network.
S105, based on the first training result and the second training result, generating a verification result corresponding to the chip to be verified.
In the method, after the neural network to be checked is obtained, a target operator consistent with the operator type of at least one original operator included in the neural network to be checked is generated, and the simulation of floating point operation in the chip to be checked by using the target operator is realized because the target data processing mode corresponding to the target operator is matched with the preset data processing mode corresponding to the operator type indicated by the design file of the chip to be checked. And then the original operator corresponding to the neural network to be checked can be replaced by the target operator, and the replaced neural network is generated. And training the neural network to be verified and the replaced neural network respectively by using the training samples to obtain a first training result and a second training result. The first training result can be determined as a reference result of the network to be checked, and the second training result can be determined as a simulation result when the chip to be checked runs the neural network to be checked. And then the first training result and the second training result are used, so that the verification result of the chip to be verified can be accurately generated, and the evaluation of the accuracy of the chip to be verified is realized.
S101 to S105 are specifically described below.
For S101:
And selecting the neural network to be checked from the constructed multiple candidate neural networks according to the training task and/or the network type. For example, training tasks may include object detection, semantic segmentation, image segmentation, keypoint detection, image classification, pose estimation, vision fashion analysis, object re-recognition, etc., and network types may include deep convolutional neural networks, self-attention networks, recurrent neural networks, etc.
For example, any Pytorch engineering can be selected from the open source projects, wherein the Pytorch engineering comprises a neural network constructed based on Pytorch, and the neural network corresponding to the Pytorch engineering is determined as the neural network to be verified.
In order to reduce the training time of the neural network and improve the efficiency of chip verification, the neural network with the network parameters smaller than the set parameter threshold value can be obtained from the constructed candidate neural networks and used as the neural network to be verified. The parameter quantity threshold value can be set according to actual requirements.
In one embodiment, the neural network to be verified may be obtained using training tasks, network types, and network parameters. For example, a plurality of candidate neural networks of different network types included under a training task may be determined based on the training task. Wherein at least one candidate neural network may be included under each network type. And selecting a candidate neural network matched with the determined network type from the plurality of candidate neural networks. And finally, selecting the neural network with the network parameter smaller than the parameter threshold value from the candidate neural networks matched with the determined network types as the neural network to be checked.
For S102:
After the neural network to be checked is obtained, the operator type of the original operator included in the neural network to be checked can be determined according to the network structure of the neural network to be checked. The original operators may include convolution operators, full join operators, pooling operators, and the like. A target operator consistent with the operator type may then be generated for the operator type of the at least one original operator. And the target data processing mode corresponding to the target operator is matched with the preset data processing mode corresponding to the operator type indicated by the design file of the chip to be tested. The preset data processing mode can be a processing mode of an operation circuit for executing the operator type operation process on the chip to be tested.
For example, the target operator consistent with the operator type may be generated for each operator type of the original operators included in the neural network to be verified, or the target operator consistent with the operator type may be generated for a part of the original operators in the various original operators included in the neural network to be verified. For example, when the performance of the to-be-verified neural network running on the chip to be verified is required to be verified, a target operator corresponding to each operator can be generated according to the operator type of each original operator. When the performance of the local original operator in the neural network to be verified is required to be verified, the target operator corresponding to the local original operator can be generated aiming at the local original operator.
In a possible implementation manner, the generating, for the operator type of at least one original operator included in the neural network to be verified, a target operator consistent with the operator type includes:
S1021, generating an initial operator corresponding to the operator type aiming at the operator type of at least one original operator included in the neural network to be checked.
S1022, testing the initial operator, and generating a test result corresponding to the initial operator.
S1023, under the condition that the test result indicates that the test is not passed, adjusting the initial operator to obtain an adjusted initial operator, taking the adjusted initial operator as a new initial operator, and returning to the step of testing the initial operator to obtain the test result until the test result of the adjusted initial operator indicates that the test is passed.
And S1024, determining the initial operator as a target operator consistent with the operator type when the test result indicates that the test is passed.
Here, for at least one operator type of the original operator, after generating an initial operator corresponding to the operator type, the initial operator may be tested. When the test fails, the initial operator can be adjusted until the test passes. The initial operator after the test is passed is determined to be the target operator consistent with the operator type, so that the performance of the generated target operator is guaranteed, and the network performance deviation between the replaced neural network and the neural network to be checked is smaller when the replaced neural network is obtained based on the target operator later. And further, based on the replaced neural network and the neural network to be verified with smaller deviation, the chip to be verified can be verified accurately.
And generating an initial operator corresponding to the operator type aiming at the operator type of at least one original operator. For example, an initial operator corresponding to at least one operator type may be generated on a unified computing device architecture (Compute Unified Device Architecture, CUDA), which may be a developed CUDA operator.
After the initial operator is generated, the initial operator can be tested, and a test result corresponding to the initial operator is generated. For example, when the initial operator is a convolution operator, dot product operation can be performed on preset data by using the initial operator, so as to generate an operation result. Judging whether the operation result is matched with a true value corresponding to the preset data, and if so, determining that the test result corresponding to the initial operator is passing. If the initial operator is not matched with the initial operator, determining that the test result corresponding to the initial operator is that the test is not passed.
In an optional implementation manner, the testing the initial operator and generating the test result corresponding to the initial operator may include:
and A1, inputting preset characteristic data into the initial operator to obtain first detection data corresponding to the initial operator, wherein the first detection data comprise predicted output data obtained when the preset characteristic data comprise input characteristic data and/or predicted gradient data obtained when the preset characteristic data comprise input gradient data.
And step A2, inputting the preset characteristic data into an original operator matched with the original operator to obtain second detection data corresponding to the original operator.
And A3, generating a test result corresponding to the initial operator based on the first detection data and the second detection data.
Preset feature data is acquired, which may include input feature data and/or input gradient data. The input feature data may be a multi-channel feature map. The input gradient data may be gradient data determined during the back propagation of the neural network.
The method comprises the steps of respectively inputting preset characteristic data into an initial operator and an original operator corresponding to the initial operator to obtain first detection data corresponding to the initial operator and second detection data corresponding to the original operator. And determining the deviation between the first detection data and the second detection data, and determining that the test result corresponding to the initial operator is test passing when the deviation is smaller than the set deviation threshold value, otherwise, determining that the test result corresponding to the initial operator is test failing.
Specifically, when the preset feature data includes input feature data, the input feature data is input into an initial operator and an original operator respectively, so as to obtain predicted output data corresponding to the initial operator and predicted output data corresponding to the original operator. Determining output deviation between the predicted output data corresponding to the initial operator and the predicted output data corresponding to the initial operator, and determining that the test result corresponding to the initial operator is test passing when the output deviation is smaller than a first deviation threshold value, otherwise determining that the test result corresponding to the initial operator is test failing.
When the preset characteristic data comprise input gradient data, the input gradient data are respectively input into an initial operator and an original operator to obtain prediction gradient data corresponding to the initial operator and prediction gradient data corresponding to the original operator. The predicted gradient data may include a gradient value corresponding to the weight data and/or a gradient value corresponding to the input feature map. Determining the prediction gradient data corresponding to the initial operator and the gradient deviation between the prediction gradient data corresponding to the initial operator, and determining the test result corresponding to the initial operator as test passing when the gradient deviation is smaller than a second deviation threshold value, otherwise determining the test result corresponding to the initial operator as test failing.
When the preset feature data includes input feature data and input gradient data, the first detection data and the second detection data include predicted output data and predicted gradient data. And determining the predicted output data corresponding to the initial operator, the output deviation between the predicted output data corresponding to the initial operator, the predicted gradient data corresponding to the initial operator and the gradient deviation between the predicted gradient data corresponding to the initial operator. And when the output deviation is greater than or equal to the first deviation threshold value or the gradient deviation is greater than or equal to the second deviation threshold value, determining that the test result corresponding to the initial operator is not passed.
In the above embodiment, the preset feature data are input into the initial operator and the original operator respectively, so as to obtain the first detection data corresponding to the initial operator and the second detection data corresponding to the original operator. The preset feature data may include input feature data and/or input gradient data, so that the forward computing performance and the backward computing performance of the initial operator are tested by using the preset feature data.
After the test result corresponding to the initial operator is obtained, if the test result indicates that the test is not passed, the initial operator may be adjusted, for example, the calculation mode, implementation logic, etc. of the initial operator may be adjusted to obtain an adjusted initial operator, and the adjusted initial operator is used as a new initial operator, and the step of S1022 is returned until the test result of the adjusted initial operator indicates that the test is passed.
If the test result indicates that the test passes, the initial operator is determined to be a target operator consistent with the operator type.
For S103:
And replacing the corresponding original operator in the neural network to be checked by using the target operator to generate a replaced neural network. For example, the neural network to be verified includes an original operator 1, an original operator 2, and an original operator 3, and a target operator 1 corresponding to the original operator 1, a target operator 2 corresponding to the original operator 2, and a target operator 3 corresponding to the original operator 3 can be determined. After the original operator is replaced, the generated replaced neural network comprises a target operator 1, a target operator 2 and a target operator 3.
For S104:
During implementation, training samples are obtained from the data set corresponding to the neural network to be verified, and the training samples are utilized to train the replaced neural network and the neural network to be verified respectively, so that a first training result corresponding to the neural network to be verified and a second training result corresponding to the neural network after replacement are obtained.
In order to improve the efficiency of chip verification, the data set corresponding to the neural network to be verified can be screened so as to reduce the number of samples included in the data set. And training the replaced neural network and the neural network to be checked by using training samples included in the screened data set.
The detection indexes included in the first training result and the second training result are matched. For example, the first training result and the second training result can comprise loss values, a change trend graph of the loss values, network precision, a change trend graph of the precision, network accuracy and the like.
For example, the training samples may be used to train the replaced neural network and the neural network to be checked for multiple times until the replaced neural network and the neural network to be checked converge, so as to obtain a first training result and a second training result.
In the process of training the replaced neural network, a floating point operation mode corresponding to the chip to be tested can be used, and a floating point operation mode corresponding to the training frame can also be used. Therefore, when training the replaced neural network by using the training sample to obtain the second training result corresponding to the replaced neural network, the method may include:
S1041, training the replaced neural network on the selected training frame by using the training samples according to a first floating point operation mode indicated by the training frame to obtain a first floating point training result.
S1042, training the replaced neural network on the selected training frame by using the training sample according to a second floating point operation mode used by the chip to be tested to obtain a second floating point training result.
S1043, determining at least one of the first floating point training result and the second floating point training result as a second training result corresponding to the replaced neural network.
The training frames may be selected from a plurality of training frames that have been built. For example, the training frame may be a graphics processor (graphics processing unit, GPU) based training frame, or the like. The training framework indicates a first floating point operation mode used in the training process.
In one mode, on a selected training frame, training the replaced neural network by using training samples according to a first floating point operation mode indicated by the training frame to obtain a first floating point training result. In the process of training the replaced neural network, a target operator in the replaced neural network processes input data by using a first floating point operation mode.
In another mode, on the selected training frame, training the replaced neural network by using the training sample according to a second floating point operation mode used by the chip to be tested to obtain a second floating point training result. In the process of training the replaced neural network, the target operator in the replaced neural network processes the input data by using a second floating point operation mode.
The first floating point operation mode may be, for example, performing a floating point operation according to a channel dimension of the feature map, the second floating point operation mode may be, for example, dividing the feature map into a plurality of feature blocks according to a length and a width, and then performing a floating point operation for each feature block, or setting the first floating point operation mode as a basic operation and the second floating point operation mode as a personalized operation, which is not limited in the disclosure.
And at least one of the first floating point training result and the second floating point training result can be determined to be the second training result corresponding to the replaced neural network.
In the above embodiment, the replaced neural network may be trained according to the first floating point operation method on the selected training frame, and/or the replaced neural network may be trained according to the second floating point operation method, so that the training mode is flexible, flexibility and diversity of the second training result are improved, and the first training result and the second training result are used subsequently, so that the chip to be tested may be checked more accurately.
In specific implementation, at least one of the following multiple training schemes can be utilized to train the replaced neural network and the neural network to be verified, so as to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.
Training the replaced neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, wherein the training method comprises the following steps:
and step B1, initializing the replaced neural network and the neural network to be checked.
And B2, respectively carrying out multiple times of training on the initialized replaced neural network and the neural network to be checked by using the training sample until the training times are equal to a set first time threshold value, and obtaining a first training result corresponding to the neural network to be checked and a second training result corresponding to the replaced neural network.
Here, the replaced neural network and the neural network to be verified may be initialized, so as to ensure that the initialization parameters of the replaced neural network and the neural network to be verified are consistent.
And training the initialized replaced neural network and the neural network to be checked for multiple times by using the training sample until the training times are equal to the set first time threshold value, so as to obtain a first training result and a second training result. Wherein the first time number threshold can be set as required.
In implementation, n preset epochs (representing training is performed once by using all samples in a training set) can be respectively used as training samples, and the training samples are utilized to perform complete training on the replaced neural network and the neural network to be checked, so that a first training result corresponding to the neural network to be checked and a second training result corresponding to the neural network to be checked are obtained. By the training mode, the accuracy of the replaced neural network and the neural network to be checked after the training is finished is highest, but the training process is time-consuming and the efficiency of chip checking is low.
Or training iteration epochs can be reduced, namely, part of epochs are selected from n epochs to train the replaced neural network and the neural network to be checked, and after the first time threshold value is trained, a first training result corresponding to the neural network to be checked and a second training result corresponding to the replaced neural network are obtained. And determining a verification result of the chip to be verified by using the first training result and the second training result. In the training mode, the accuracy of the neural network after replacement and the neural network to be checked after the training is completed cannot reach the highest accuracy, but the training process is time-consuming and the efficiency of chip checking is high.
Here, the training times of the neural network can be flexibly controlled by setting the first time threshold value, and the verification efficiency of the chip can be improved by reducing the first time threshold value.
Training the replaced neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, wherein the training scheme II comprises the following steps:
And step C1, initializing the neural network to be verified.
And C2, training the initialized neural network to be checked for multiple times by using a first training sample in the training samples to obtain the neural network to be checked containing the first network parameters. And determining the first network parameter as a network parameter of the post-replacement neural network.
And C3, respectively carrying out multiple times of training on the neural network to be checked containing the first network parameters and the replaced neural network containing the first network parameters by using a second training sample in the training samples to obtain a first training result corresponding to the neural network to be checked and a second training result corresponding to the replaced neural network.
And when the training times are fixed, the training time of the neural network to be checked is smaller than the training time of the neural network after replacement. Therefore, after the neural network to be checked is initialized, the neural network to be checked can be trained first, and the neural network to be checked containing the first network parameters is obtained. And then the first network parameter is used as the network parameter of the replaced neural network, and the replaced neural network is trained until the replaced neural network converges, so that the time required for reaching the highest precision of the replaced neural network can be reduced, and the training efficiency of the replaced neural network is improved.
In the specific implementation, a first training sample in the training samples is utilized to train the initialized neural network to be checked for multiple times until the pre-training cut-off condition is met, and a first model file corresponding to the neural network to be checked is obtained. The first model file indicates a neural network to be verified, which contains first network parameters. The pre-training cut-off condition can be set according to actual requirements.
By way of example, the pre-training cutoff condition may include at least one of a first, set number of exercises, such as after training 100 epochs, determining that the pre-training cutoff condition is met. 2. The loss value of the trained neural network to be checked is smaller than a first preset value. 3. The accuracy of the trained neural network to be checked is larger than a second preset value.
And when the method is implemented, replacing the initial operator in the neural network to be verified, which is indicated by the first model file and contains the first network parameters, by using the target operator, wherein the replaced neural network contains the first network parameters. And training the to-be-checked neural network containing the first network parameters and the replaced neural network containing the first network parameters for multiple times by a second training sample in the training samples until the to-be-checked neural network and the replaced neural network converge or until the training times are equal to a preset number of times, so as to obtain a first training result corresponding to the to-be-checked neural network and a second training result corresponding to the replaced neural network. Wherein the first training sample and the second training sample may be at least some of the training samples.
In the above embodiment, after the to-be-verified neural network including the first network parameter is obtained, the first network parameter may be used as a network parameter of the post-replacement neural network, and the training sample is used to train the post-replacement neural network including the first network parameter multiple times, so as to obtain a second training result corresponding to the post-replacement neural network. Through the mode, when the training times are fixed, the accuracy of the second training result can be improved. When the accuracy is fixed, the time period consumed for obtaining the second training result can be reduced.
Training the replaced neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network, wherein the training method comprises the following steps of:
and D1, initializing the replaced neural network and the neural network to be checked.
And D2, respectively training the initialized replaced neural network and the neural network to be checked for multiple times by using a first training sample in the training samples until the training times are equal to a set second time threshold value, so as to obtain the replaced neural network containing the second network parameters and the neural network to be checked containing the third network parameters.
And D3, respectively carrying out multiple times of training on the neural network to be verified containing the second network parameters and the neural network to be verified containing the third network parameters by using a second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.
In the implementation, the replaced neural network and the neural network to be checked can be initialized, so that the network parameters of the replaced neural network and the neural network to be checked after the initialization are consistent. And respectively carrying out multiple times of training on the initialized replaced neural network and the neural network to be checked by using a first training sample in the training samples until the training times are equal to a set second time threshold value to obtain the replaced neural network containing the second network parameters and the neural network to be checked containing the third network parameters.
And when the training times are fixed, the training time of the neural network to be checked is smaller than the training time of the neural network after replacement. Thus, a neural network to be verified comprising the second network parameter and a neural network to be verified comprising the third network parameter may be generated. And respectively carrying out multiple times of training on the neural network to be verified containing the second network parameters and the neural network to be verified containing the third network parameters by using a second training sample in the training samples until the neural network to be verified and the replaced neural network converge or until the training times are equal to a preset number of times, so as to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network. The second training result corresponding to the replaced neural network is obtained by training the neural network to be verified, which contains the second network parameters.
Because the second network parameter and the third network parameter are different in value, deviation exists between the first training result and the second training result obtained in the subsequent training process, for example, the accuracy included in the first training result is different from the accuracy included in the second training result, so that a verification result corresponding to the chip to be verified can be generated by comparing the accuracy of the first training result with the accuracy of the second training result.
For S105:
in implementation, a verification result corresponding to the chip to be verified can be generated based on the first training result and the second training result. For example, when the first training result and the second training result include the precision, a difference between the precision in the first training result and the precision in the second training result can be determined, and when the difference is smaller than or equal to a deviation threshold value, the verification result corresponding to the chip to be tested can be determined that the precision performance of the chip to be tested is good. When the difference value is larger than the deviation threshold value, the verification result corresponding to the chip to be tested is determined to be that the accuracy performance of the chip to be tested is poor.
Under the condition that the second training result comprises the first floating point training result and the second floating point training result, generating a verification result corresponding to the chip to be verified based on the first training result and the second training result can comprise any one of the following modes:
In the first mode, when a first deviation value between a first floating point training result in the second training result and the first training result is smaller than or equal to a set first threshold value, and a second deviation value between the first floating point training result and a second floating point training result in the second training result is smaller than or equal to a set second threshold value, a verification result indicating that the chip to be verified passes is generated.
Here, when the first deviation value between the first floating point training result and the first training result is smaller than or equal to a set first threshold value, and the second deviation value between the first floating point training result and the second floating point training result is smaller than or equal to a set second threshold value, it is determined that the floating point number operation mode (such as the second floating point operation mode) used by the chip to be tested is better, and the performance corresponding to the chip to be tested is better, and the generated verification result may be that the chip to be tested passes the verification.
And in the second mode, under the condition that a first deviation value between the first floating point training result and the first training result is larger than a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is smaller than or equal to a set second threshold value, generating a verification result indicating that the design file of the chip to be verified is to be adjusted.
When the first deviation value between the first floating point training result and the first training result is larger than a first threshold value, the performance corresponding to the chip to be tested is determined to be poor, and when the second deviation value between the first floating point training result and the second floating point training result is smaller than or equal to a set second threshold value, the floating point number operation mode (such as a second floating point operation mode) used by the chip to be tested is determined to be better, the generated verification result can be that the design file of the chip to be tested is to be adjusted.
And generating a verification result indicating that a floating point number operation mode used by the chip to be verified is to be adjusted under the condition that a first deviation value between the first floating point training result and the first training result is smaller than or equal to a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value.
When the second deviation value between the first floating point training result and the second floating point training result is larger than the second threshold value, the floating point number operation mode (such as the second floating point operation mode) used by the chip to be tested is determined to be worse, and the generated verification result can be the floating point number operation mode used by the chip to be tested to be adjusted.
And in the fourth mode, under the condition that the first deviation value between the first floating point training result and the first training result is larger than a set first threshold value and the second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value, generating a floating point number operation mode used for indicating the chip to be tested and a verification result to be adjusted of the design file.
When the first deviation value between the first floating point training result and the first training result is larger than a first threshold value and the second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value, it is determined that the floating point number operation mode used by the chip to be tested is poor and the corresponding performance of the chip to be tested is poor, and the generated verification result can be the floating point number operation mode used by the chip to be tested and the design file to be adjusted.
Here, through setting up multiple mode, the check result of chip that waits to test is more accurate and more appearance.
In the specific implementation, each training detail of the neural network training process to be verified and each training detail of the neural network training process after replacement can be recorded in the process of verifying the chip to be verified, for example, the training details can comprise training engineering introduction, operator list, training method, training process and the like. After the chip to be tested flows, the chip obtained by the flow can be tested according to each recorded training detail to obtain a test result, and the test result can be compared with the first training result and the second training result to determine the problems, the performances and the like of the chip.
It will be appreciated by those skilled in the art that in the above-described method of the specific embodiments, the written order of steps is not meant to imply a strict order of execution but rather should be construed according to the function and possibly inherent logic of the steps.
Based on the same concept, the embodiment of the disclosure further provides a chip verification device, referring to fig. 2, which is a schematic architecture diagram of the chip verification device provided by the embodiment of the disclosure, including an obtaining module 201, a first generating module 202, a second generating module 203, a training module 204, and a third generating module 205, specifically:
an acquisition module 201, configured to acquire a neural network to be checked;
A first generation module 202, configured to generate, for an operator type of at least one original operator included in the neural network to be verified, a target operator consistent with the operator type;
the second generating module 203 is configured to replace a corresponding original operator in the neural network to be verified with the target operator, so as to generate a replaced neural network;
The training module 204 is configured to train the replaced neural network and the neural network to be verified by using training samples, so as to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network;
And a third generating module 205, configured to generate a verification result corresponding to the chip to be tested based on the first training result and the second training result.
In a possible implementation manner, the first generation module 202 is configured to, when generating, for at least one operator type of an original operator included in the neural network to be verified, a target operator consistent with the operator type:
Generating an initial operator corresponding to at least one operator type of the original operator included in the neural network to be checked;
testing the initial operator to generate a test result corresponding to the initial operator;
The method comprises the steps of obtaining a test result by taking an initial operator after adjustment as a new initial operator, and returning to the step of testing the initial operator until the test result of the initial operator after adjustment indicates that the test is passed;
And determining the initial operator as a target operator consistent with the operator type in the condition that the test result indicates that the test is passed.
In a possible implementation manner, the first generating module 202 is configured to, when testing an initial operator and generating a test result corresponding to the initial operator:
Inputting preset characteristic data into the initial operator to obtain first detection data corresponding to the initial operator, wherein the first detection data comprises predicted output data obtained when the preset characteristic data comprises input characteristic data and/or predicted gradient data obtained when the preset characteristic data comprises input gradient data, and
Inputting the preset characteristic data to an original operator matched with the original operator to obtain second detection data corresponding to the original operator;
and generating a test result corresponding to the initial operator based on the first detection data and the second detection data.
In a possible implementation manner, the training module 204 is configured to, when training the post-replacement neural network by using the training sample to obtain a second training result corresponding to the post-replacement neural network:
training the replaced neural network on the selected training frame by utilizing the training sample according to a first floating point operation mode indicated by the training frame to obtain a first floating point training result and/or,
Training the replaced neural network on the selected training frame by utilizing the training sample according to a second floating point operation mode used by the chip to be tested to obtain a second floating point training result;
And determining at least one of the first floating point training result and the second floating point training result as a second training result corresponding to the replaced neural network.
In a possible implementation manner, the third generating module 205 is configured to, when generating the verification result corresponding to the chip under test based on the first training result and the second training result:
Generating a verification result indicating that the chip to be verified passes under the condition that a first deviation value between a first floating point training result in the second training results and the first training result is smaller than or equal to a set first threshold value and a second deviation value between the first floating point training result and a second floating point training result in the second training results is smaller than or equal to a set second threshold value;
generating a verification result indicating that the design file of the chip to be verified is to be adjusted under the condition that a first deviation value between the first floating point training result and the first training result is larger than a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is smaller than or equal to a set second threshold value;
Generating a verification result indicating that a floating point number operation mode used by the chip to be verified is to be adjusted under the condition that a first deviation value between the first floating point training result and the first training result is smaller than or equal to a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value;
And under the condition that a first deviation value between the first floating point training result and the first training result is larger than a set first threshold value and a second deviation value between the first floating point training result and the second floating point training result is larger than a set second threshold value, generating a floating point number operation mode used for indicating the chip to be tested and a verification result to be adjusted of the design file.
In a possible implementation manner, the training module 204 is configured to, when training the post-replacement neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the post-replacement neural network, respectively:
Initializing the replaced neural network and the neural network to be checked;
And respectively carrying out multiple times of training on the initialized replaced neural network and the neural network to be checked by utilizing the training sample until the training times are equal to the set first time threshold value, and obtaining a first training result corresponding to the neural network to be checked and a second training result corresponding to the replaced neural network.
In a possible implementation manner, the training module 204 is configured to, when training the post-replacement neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the post-replacement neural network, respectively:
initializing the neural network to be checked;
training the initialized neural network to be checked for multiple times by using a first training sample in the training samples to obtain a neural network to be checked containing first network parameters;
And respectively carrying out multiple times of training on the neural network to be verified containing the first network parameters and the replaced neural network containing the first network parameters by using a second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.
In a possible implementation manner, the training module 204 is configured to, when training the post-replacement neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the post-replacement neural network, respectively:
Initializing the replaced neural network and the neural network to be checked;
Respectively training the initialized replaced neural network and the neural network to be checked for multiple times by using a first training sample in the training samples until the training times are equal to a set second time threshold value, so as to obtain the replaced neural network containing the second network parameters and the neural network to be checked containing the third network parameters;
And respectively carrying out multiple times of training on the neural network to be verified containing the second network parameters and the neural network to be verified containing the third network parameters by using a second training sample in the training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network.
In some embodiments, the functions or templates included in the apparatus provided by the embodiments of the present disclosure may be used to perform the methods described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
Based on the same technical concept, the embodiment of the disclosure also provides electronic equipment. Referring to fig. 3, a schematic structural diagram of an electronic device according to an embodiment of the disclosure includes a processor 301, a memory 302, and a bus 303. The memory 302 is configured to store execution instructions, including a memory 3021 and an external memory 3022, where the memory 3021 is also referred to as an internal memory, and is configured to temporarily store operation data in the processor 301 and data exchanged with the external memory 3022, such as a hard disk, where the processor 301 exchanges data with the external memory 3022 via the memory 3021, and when the electronic device 300 is running, the processor 301 and the memory 302 communicate with each other via the bus 303, so that the processor 301 executes the following instructions:
acquiring a neural network to be checked;
Generating a target operator consistent with the operator type aiming at the operator type of at least one original operator included in the neural network to be checked;
replacing the corresponding original operator in the neural network to be verified by using the target operator to generate a replaced neural network;
training the replaced neural network and the neural network to be verified by using training samples to obtain a first training result corresponding to the neural network to be verified and a second training result corresponding to the replaced neural network;
And generating a verification result corresponding to the chip to be verified based on the first training result and the second training result.
Furthermore, the embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, which when executed by a processor performs the steps of the chip authentication method described in the above-described method embodiments. Wherein the storage medium may be a volatile or nonvolatile computer readable storage medium.
The embodiments of the present disclosure further provide a computer program product, where the computer program product carries program code, where instructions included in the program code may be used to perform the steps of the chip verification method described in the foregoing method embodiments, and specifically reference may be made to the foregoing method embodiments, which are not described herein in detail.
Wherein the above-mentioned computer program product may be realized in particular by means of hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present disclosure may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in essence or a part contributing to the prior art or a part of the technical solution, or in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present disclosure. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
The foregoing is merely a specific embodiment of the disclosure, but the protection scope of the disclosure is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the disclosure, and it should be covered in the protection scope of the disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.