Disclosure of Invention
In view of the above, the present application aims to provide a smart home control method, a smart home control system and a storage medium, so as to improve the security of smart home control.
The first technical scheme adopted by the application is as follows:
an intelligent home control method, comprising:
collecting a voice signal to be recognized;
performing voice recognition on the voice signal to be recognized, and extracting keywords of the voice signal to be recognized;
Acquiring a control instruction corresponding to the keyword, and acquiring a list of authorized users corresponding to the control instruction;
performing voiceprint recognition on the voice signal to be recognized, and extracting voice voiceprint characteristics of the voice signal to be recognized;
Determining that the authorized user list does not comprise strangers, and acquiring first voiceprint features of authorized users corresponding to the control instruction; performing feature matching on the first voiceprint feature and the voice voiceprint feature to obtain a first correlation coefficient; determining that the first correlation coefficient is larger than a matching threshold value, and generating an intelligent home control signal according to the keyword;
determining that the authorized user list comprises strangers, and acquiring second voice characteristics of the unauthorized users corresponding to the control instruction; performing feature matching on the second voiceprint feature and the voice voiceprint feature to obtain a second correlation number; determining that the second correlation coefficient is smaller than a matching threshold value, and generating an intelligent home control signal according to the keyword;
Wherein the stranger is a user who does not have voice print features recorded.
Further, the step of performing voiceprint recognition on the voice signal to be recognized and extracting the voice voiceprint feature of the voice signal to be recognized specifically includes:
performing high-pass filtering on the voice signal to be recognized to obtain a high-pass voice signal;
framing the high-pass voice signal to obtain a high-pass voice signal frame;
Multiplying the high-pass voice signal frame by a hamming window to obtain a hamming voice signal frame;
Performing fast Fourier transform on the Hamming voice signal frame to obtain a Hamming voice signal frame frequency spectrum;
Performing triangular band-pass filtering on the Hamming voice signal frame frequency spectrum to obtain a triangular voice signal frame frequency spectrum;
performing discrete cosine transform on the triangular voice signal frame frequency spectrum to obtain a Mel frequency cepstrum coefficient;
and extracting voice voiceprint features of the voice signal to be recognized according to the Mel frequency cepstrum coefficient.
Further, the step of collecting the voice signal to be recognized specifically includes:
Collecting a wake-up voice signal;
And determining that the wake-up voice signal comprises wake-up words, and collecting the voice signal to be recognized.
Further, the step of collecting the voice signal to be recognized specifically includes:
Collecting a sound signal to be identified;
Carrying out voice framing on the voice signal to be identified to obtain a plurality of voice signal frames to be identified;
Calculating the square sum of the voice signal frames to be identified to obtain short-time energy of the voice signal frames to be identified;
and determining that the short-time energy is larger than an energy threshold value, and collecting the voice signal to be recognized.
Further, the step of collecting the voice signal to be recognized specifically includes:
Collecting a sound signal to be identified;
Carrying out voice framing on the voice signal to be identified to obtain a plurality of voice signal frames to be identified;
calculating the short-time average zero-crossing rate of the voice signal frame to be identified;
and determining that the short-time average zero crossing rate is larger than a zero crossing rate threshold value, and collecting the voice signal to be recognized.
Further, the first correlation coefficient is a pearson correlation coefficient, and the second correlation coefficient is a pearson correlation coefficient.
Further, the step of collecting the voice signal to be recognized specifically includes:
collecting unprocessed voice signals to be recognized;
And denoising the unprocessed voice signal to be recognized to obtain the voice signal to be recognized.
The second technical scheme adopted by the application is as follows:
An intelligent home control system, comprising:
The voice acquisition module is used for acquiring a voice signal to be identified;
the voiceprint recognition module is used for carrying out voiceprint recognition on the voice signal to be recognized and extracting voice voiceprint characteristics of the voice signal to be recognized;
The voice recognition module is used for carrying out voice recognition on the voice signal to be recognized and extracting keywords of the voice signal to be recognized;
the permission acquisition module is used for acquiring a control instruction corresponding to the keyword and acquiring a permission user list corresponding to the control instruction;
The authority matching module is used for determining that the authority user list does not comprise strangers and acquiring first voiceprint features of the authority users corresponding to the control instructions; performing feature matching on the first voiceprint feature and the voice voiceprint feature to obtain a first correlation coefficient; determining that the first correlation coefficient is larger than a matching threshold value, and generating an intelligent home control signal according to the keyword; determining that the authorized user list comprises strangers, and acquiring second voice characteristics of the unauthorized users corresponding to the control instruction; performing feature matching on the second voiceprint feature and the voice voiceprint feature to obtain a second correlation number; determining that the second correlation coefficient is smaller than a matching threshold value, and generating an intelligent home control signal according to the keyword;
Wherein the stranger is a user who does not have voice print features recorded.
The third technical scheme adopted by the application is as follows:
An intelligent home control system, comprising:
at least one processor;
at least one memory for storing at least one program;
And when the at least one program is executed by the at least one processor, the at least one processor is enabled to realize the intelligent home control method.
The fourth technical scheme adopted by the application is as follows:
A computer readable storage medium having stored thereon a computer program which when executed by a processor implements the smart home control method.
According to the embodiment of the application, the voice signal to be recognized is collected to perform voice recognition and voiceprint recognition, the key words and the voiceprint features of the voice signal to be recognized are obtained, the corresponding control command and the input voiceprint features corresponding to the control command are obtained according to the key words, the voiceprint features of the voice signal and the input voiceprint features are subjected to feature matching, whether the control authority exists in the voice signal to be recognized is judged, and the intelligent home control signal is generated after the control authority exists in the voice signal to be recognized is determined. Compared with the existing intelligent home control method, the method can identify whether the user sending the voice signal has the operation authority or not, and has higher safety.
Detailed Description
The conception, specific structure, and technical effects produced by the present application will be clearly and completely described below with reference to the embodiments and the drawings to fully understand the objects, aspects, and effects of the present application.
The application will now be described in further detail with reference to the drawings and to specific examples. The step numbers in the following embodiments are set for convenience of illustration only, and the order between the steps is not limited in any way, and the execution order of the steps in the embodiments may be adaptively adjusted according to the understanding of those skilled in the art. Furthermore, for several of the embodiments described below, it is denoted as at least one.
It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element of the same type from another. For example, a first element could also be termed a second element, and, similarly, a second element could also be termed a first element, without departing from the scope of the present disclosure. The use of any and all examples, or exemplary language (e.g., "such as") provided herein, is intended merely to better illuminate embodiments of the application and does not pose a limitation on the scope of the application unless otherwise claimed.
The current intelligent home terminal collects voice of a user and carries out voice recognition, control information in the voice is extracted through the voice recognition, the control information is transmitted to the processor, the processor generates a corresponding control signal according to the control information, so that intelligent home voice control is achieved, but the current intelligent home terminal cannot carry out voiceprint recognition, so that all users can control the intelligent home terminal, and if a child does not have a guardian to accompany, potential safety hazards are easily caused.
As shown in fig. 1, an embodiment of the present application provides an intelligent home control method, including:
S110, collecting a voice signal to be recognized;
S120, performing voice recognition on the voice signal to be recognized, and extracting keywords of the voice signal to be recognized;
s130, acquiring a control instruction corresponding to the keyword, and acquiring a list of authorized users corresponding to the control instruction;
s140, carrying out voiceprint recognition on the voice signal to be recognized, and extracting voice voiceprint characteristics of the voice signal to be recognized;
S150, determining that the authorized user list does not include strangers, and acquiring first voiceprint features of authorized users corresponding to the control instruction; performing feature matching on the first voiceprint feature and the voice voiceprint feature to obtain a first correlation coefficient; determining that the first correlation coefficient is larger than a matching threshold value, and generating an intelligent home control signal according to the keyword;
S160, determining that the authorized user list comprises strangers, and acquiring second voice characteristics of the unauthorized users corresponding to the control instruction; performing feature matching on the second voiceprint feature and the voice voiceprint feature to obtain a second correlation number; and determining that the second correlation coefficient is smaller than a matching threshold value, and generating an intelligent home control signal according to the keywords.
In the control process of the intelligent home, the intelligent home terminal can collect the voice signal to be recognized, and carry out voiceprint recognition and voice recognition on the voice signal to be recognized. Generally, keyword recognition and extraction of a voice signal to be recognized are performed first, after keywords are obtained through recognition and corresponding control instructions are obtained, authorized users corresponding to the control instructions, namely users with authorization to the operation instructions, need to perform voiceprint recognition on the voice signal to be recognized to obtain voice voiceprint characteristics of the voice signal to be recognized. When the authorized user comprises strangers, the forbidden object of the control instruction is some members of family members, such as children and old people, and at the moment, the control instruction can be executed by determining that the unauthorized user who sends the control instruction is not the unauthorized user by performing feature matching on voice characteristics and voice characteristics corresponding to the children and the old people in the family members, namely, second voice characteristics of the unauthorized user. When the authorized user does not include strangers, the control instruction is described as a limiting control instruction, and only some members in the family can execute the control instruction, at the moment, the voice voiceprint characteristics are only required to be matched with the voiceprint characteristics of the authorized user in the family members, namely, the first voiceprint characteristics of the authorized user, and the authorized user can execute the control instruction after the control instruction is determined to be issued.
In the process of voiceprint recognition of a voice signal to be recognized, voiceprint feature extraction can be performed on the voice signal to be recognized by using mel cepstrum coefficients in consideration of safety and recognition speed. Compared with linear cepstrum analysis, the Mel cepstrum coefficient is not dependent on the hypothesis of a voice generation model, so that the voice print characteristics irrelevant to the text can be better extracted, and the requirement of being not constrained by a specific command is met. And after the feature extraction, carrying out matching verification on the identity in the database according to a related feature matching algorithm. The feature extraction needs to reduce the influence of information irrelevant to recognition in the voice signal, reduce the data volume to be processed in the subsequent recognition stage, extract basic features capable of representing people from the voice signal, the extracted features must be capable of effectively distinguishing different users, and keep relatively stable to voice change of the same user. The voice characteristic parameter extraction firstly needs to pass a voice signal to be recognized through a high-pass filter to obtain the high-pass voice signal, and the high-pass filter is used for reducing information irrelevant to voiceprint recognition in the voice signal to be recognized and reducing the data quantity in the subsequent recognition stage. By raising the high frequency part, the spectrum of the speech signal to be recognized is flattened. Meanwhile, the influence of lips and vocal cords in the sounding process is eliminated, and the high-frequency part of the voice signal restrained by the sounding system is compensated. And then 256 sampling points are integrated into an observation unit to acquire a high-pass voice signal frame. Since the input of the fourier transform requirement is stationary, the sound signal is changed into a stationary signal frame by frame processing, so that the distribution of the lattice frequency components is obtained when the fourier transform is performed. And multiplying the high-pass voice signal frame by a Hamming window to obtain the Hamming voice signal frame, and increasing the continuity of the left end and the right end of the frame by windowing to reduce the Gibbs effect so that the overall situation is more continuous. And then each frame is subjected to fast Fourier transform to obtain energy distribution on a frequency spectrum, wherein the fast Fourier transform has the following formula:
Where S (k) is a hamming speech signal frame image function after the fast fourier transform, S (n) is a hamming speech signal frame, and DFT [ ] is the fast fourier transform. After the frame image function of the hamming voice signal is obtained, the energy distribution of s (k), namely the frame frequency spectrum of the hamming voice signal, can be obtained, wherein the energy distribution formula of s (k) is as follows:
where p (k) is the hamming voice signal frame spectrum after the fast fourier transform. After the Hamming voice signal frame frequency spectrum is obtained, triangular band-pass filtering can be carried out on the Hamming voice signal frame frequency spectrum to obtain the triangular voice signal frame frequency spectrum, the frequency spectrum can be smoothed through the triangular band filtering, meanwhile, frequency spectrum harmonic waves are eliminated, formants of voice are highlighted, and subsequent operation quantity is reduced. Discrete cosine transform is carried out on the frequency spectrum of the triangular voice signal frame, and the L-order Mel frequency cepstrum coefficient is obtained. The formula for linear frequency conversion to mel-frequency cepstral coefficients is as follows:
After obtaining the mel frequency cepstrum coefficient, since the standard mel frequency cepstrum coefficient only reflects the static characteristics of the voice signal to be recognized, the dynamic characteristics of the voice signal to be recognized can be described by the differential spectrum of the static characteristics. The static features and the dynamic features of the voice signal to be recognized are the voice voiceprint features of the voice signal to be recognized.
After the voice voiceprint characteristics of the voice signal to be recognized are extracted, voiceprint matching is required to be carried out on the voice voiceprint characteristics and the first voiceprint characteristics of the authorized user or the second voiceprint characteristics of the unauthorized user. Different from the conventional voiceprint recognition and the matching of all users in the database, the method and the device acquire the voiceprint data of the users with or without the authority under the current instruction from the database according to different conditions according to whether the authorized users of the controlled home contain strangers or not, so that the problem that a great deal of time is required for matching when the data stored in the database is excessive is avoided. And meanwhile, the method is only matched with the authority user under the corresponding command, so that the time consumption is further reduced. The pearson correlation coefficient can be adopted for voiceprint matching, and the feature with the largest pearson correlation coefficient is selected as the feature for matching. If the correlation coefficient is smaller than the set threshold, judging that the voiceprint is not in the matching range; if the correlation coefficient is larger than the set threshold, the user with the characteristic of the maximum pearson correlation coefficient is taken as the matching person of the voiceprint. The pearson correlation coefficient calculation formula is:
Wherein x is a voice voiceprint feature, y is a first voiceprint feature or a second voiceprint feature,Is the average value of voice voiceprint characteristics,/>Is the average of the first voiceprint feature or the average of the second voiceprint feature.
In practical use, it is obviously not reasonable to process all voice signals in real time. Therefore, in order to save hardware and software resources, activity detection is required for the voice signal to ensure that the input detected voice signal is useful information. And detecting the voice activity by adopting a method of combining short-time energy and short-time average zero crossing rate with double thresholds. The short-time energy is simple to calculate, the contrast ratio to sound is high, the short-time average zero-crossing rate is good in a noise-free position, the advantages of the short-time average zero-crossing rate and the short-time average zero-crossing rate are combined, the constraint characteristic of the double thresholds is achieved, the error rate can be reduced, and the consumption of calculation resources can be reduced. The speech signal is first framed, taking a frame of 20ms, where the input signal sampling rate is 8000HZ. Each frame is 160samples in length. And then, calculating the square sum of the intra-frame signals to obtain short-time energy. And translating all samples in the signal frame by 1, multiplying corresponding points, and carrying out zero crossing at the position when the sign is negative, wherein the short-time average zero crossing rate of the frame is obtained only by solving the number of products of all negative numbers in the frame. The energy threshold and the zero crossing rate threshold may be obtained by neural network training. And setting two threshold values for the short-time energy and the short-time average zero-crossing rate respectively, calculating the short-time energy and the short-time average zero-crossing rate of each frame, and judging the voice start when the short-time energy and the short-time average zero-crossing rate exceed the threshold values.
Offline wakeup is an important entry for voice interaction, and the main difficulty is that the contradiction between rapid and accurate recognition and low power consumption requirements can be balanced, and furthermore, the real-time performance of offline wakeup requires that the wakeup system is always in operation. The intelligent home control method uses an open-source wake-up word detection engine Snowboy, and Snowboy supports custom wake-up words, so that model training can be conducted on training words set by users in an open mode, and the effect of rapid and accurate wake-up is achieved. The use Snowboy has the advantages of no need of connecting to a network, low resource consumption, high customization and the like. The construction of the wake-up word requires recording three wake-up word recordings on the terminal; uploading the record to Snowboy official network for training, and adjusting sensitivity according to the requirement; after training, the trained wake-up word model is downloaded to the terminal. For example: the key word "small Bai Xiaobai" needs to be built on the raspberry group terminal, and first install Snowboy is compiled using the make command after the relevant environment of Snowboy is installed on the raspberry group. Then, the keyword 'small Bai Xiaobai' is recorded three times on the terminal, the recording needs to be in a relatively quiet environment, and the recorded file is saved as a wav file. And finally logging in Snowboy official networks, uploading three sound recording files, waiting for training, testing on the terminal, adjusting the sensitivity according to actual conditions, downloading the trained pmdl model file into the terminal, and replacing the original model. The wake word has been changed to "small Bai Xiaobai" so far.
The embodiment of the application also provides an intelligent home control system, which comprises:
The voice acquisition module is used for acquiring a voice signal to be identified;
the voiceprint recognition module is used for carrying out voiceprint recognition on the voice signal to be recognized and extracting voice voiceprint characteristics of the voice signal to be recognized;
The voice recognition module is used for carrying out voice recognition on the voice signal to be recognized and extracting keywords of the voice signal to be recognized;
the permission acquisition module is used for acquiring a control instruction corresponding to the keyword and acquiring a permission user list corresponding to the control instruction;
The authority matching module is used for determining that the authority user list does not comprise strangers and acquiring first voiceprint features of the authority users corresponding to the control instructions; performing feature matching on the first voiceprint feature and the voice voiceprint feature to obtain a first correlation coefficient; determining that the first correlation coefficient is larger than a matching threshold value, and generating an intelligent home control signal according to the keyword; determining that the authorized user list comprises strangers, and acquiring second voice characteristics of the unauthorized users corresponding to the control instruction; performing feature matching on the second voiceprint feature and the voice voiceprint feature to obtain a second correlation number; and determining that the second correlation coefficient is smaller than a matching threshold value, and generating an intelligent home control signal according to the keywords.
The intelligent home control system can comprise a voice activity detection module, an offline wake-up module, a voice recognition module, a voiceprint recognition module permission matching module and an instruction execution module. After the user speaks the wake-up word, the terminal is activated through the offline wake-up module. And then, the terminal records the voice signal to be recognized sent by the user, and the voice signal to be recognized is transmitted to the voice recognition module and the voiceprint recognition module for processing. The voice recognition module converts voice into characters and then extracts keywords; and uploading the voice signal to be recognized of the user to a server by the voiceprint recognition module, carrying out voiceprint recognition by the server, extracting voiceprint characteristics and comparing the voiceprint characteristics with voiceprint characteristics in a database, and returning a comparison result. And the terminal judges whether the user has authority to control the home according to the voiceprint recognition return result, if so, the terminal executes the instruction to control the corresponding home, and if not, the terminal does not execute the instruction.
The construction of the system can comprise a user interaction interface, a raspberry group terminal, a server, a voiceprint recognition algorithm, hundred-degree intelligent cloud real-time voice recognition and various household devices.
The intelligent home control system adopts the raspberry pie as the core control module, and the raspberry pie has the advantages of being adaptive to various deep learning frameworks, complete in function, various in module types, low in power consumption, moderate in operation speed and the like. Raspberry pie is used for interaction with the handset APP and receives audio input data from the microphone while outputting audio data to the speaker, also controlling ESP8266, etc. In addition, the raspberry group also serves as a software platform to complete processing of voice signals so as to control the home.
The intelligent home control system adopts RESPEAKER microphone modules as an audio input mode, and RESPEAKER microphones are connected with pins 1 to 40 of the raspberry group. The recorded audio file will be stored locally in the raspberry group in preparation for subsequent speech recognition and voiceprint recognition. Compared with a common USB microphone, the RESPEAKER microphone module has the characteristics of good noise reduction effect, long sound receiving distance, good sound pickup effect and the like.
The intelligent home control system adopts a mobile phone APP as a medium of a user graphical interactive interface, the mobile phone APP is developed by using JAVA, communication is carried out between the mobile phone APP and a raspberry group through a Socket protocol, and a user can carry out user management, home management, authority setting and other operations through the mobile phone APP. The user can realize operations such as user management, home management, authority setting and the like through the mobile phone APP. The user opens an APP pre-installed on the mobile phone, enters a login page, and inputs a user name and a password; if the password is correct, entering a menu page; if the password is wrong, prompting the password to be wrong. The menu page has three options of user management, home management and authority setting, and clicking the option can enter the page corresponding to the option. The user can perform operations of newly building and deleting the user on the user management page. When a new user is created, the user inputs name information and then clicks "record". Before recording, the mobile phone terminal sends a signal to the intelligent home terminal, and the intelligent home terminal starts a microphone after receiving the signal; meanwhile, the mobile phone end prompts the user to record voice information by using the intelligent home terminal. And after the recording is completed, the intelligent home terminal uploads the voice information and the user information to a server, the server extracts voiceprint features of the voice information, and the extracted feature data and the user name are stored in a database. For the user who has input information, the user name is displayed in a list form, the user clicks the user name, the page jumps out of the user ' whether to delete the user ' and the user can be deleted if the page clicks the user ' yes; clicking "no" does not react. After the setting is completed, the mobile phone terminal sends information to the intelligent home terminal, the intelligent home terminal analyzes the information and sends the information to the server, and the server deletes the voiceprint feature data of the user. The user can add and delete home operations on the home management page. After the setting is completed, the mobile phone terminal sends information to the intelligent home terminal, and home information of the mobile phone terminal and the intelligent home terminal is updated; the user can set the use authority of the entered home for the entered user on the 'set authority' page. After the setting is completed, the mobile phone terminal sends information to the intelligent home terminal, the intelligent home terminal analyzes the information and sends the information to the server, and the server updates the user name and the authority for controlling different home furnishings to the database.
The intelligent home control system adopts a loudspeaker as an audio amplification output mode. The loudspeaker has two wiring ports, and the 3.5mm audio output interface of one end connection raspberry group obtains the audio signal that the raspberry group outputted, and the USB port of the other end connection raspberry group supplies power to the loudspeaker.
The intelligent home control system can communicate with the home equipment by adopting an ESP-12F WIFI module. The module core processor ESP 8236 integrates the TENSILICA L ultra-low power consumption 32-bit micro MCU of the lead TENSILICA L in the industry in a smaller-size package, has a 16-bit simplified mode, supports the standard IEEE802.11 b/g/n protocol and a complete TCP/IP protocol stack, and can be used for adding networking functions for the existing equipment. When the ESP8266 module is electrified and is accessed to a preset local area network, current IP information is fed back through a serial port. The raspberry group can control the state of the relevant pins by accessing the IP and adding a pre-written command, thereby controlling the switching state of the smart home.
The content in the method embodiment is applicable to the system embodiment, the functions specifically realized by the system embodiment are the same as those of the method embodiment, and the achieved beneficial effects are the same as those of the method embodiment.
The embodiment of the application also provides an intelligent home control system, which comprises:
at least one processor;
at least one memory for storing at least one program;
And when the at least one program is executed by the at least one processor, the at least one processor is enabled to realize the intelligent home control method.
The content in the method embodiment is applicable to the system embodiment, the functions specifically realized by the system embodiment are the same as those of the method embodiment, and the achieved beneficial effects are the same as those of the method embodiment.
In addition, the embodiment of the present application further provides a storage medium, in which processor executable instructions are stored, where the processor executable instructions are used for executing steps of an interactive information processing method according to any one of the foregoing method embodiments when the processor executable instructions are executed by a processor. For the storage medium, it may include high-speed random access memory, but may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. It can be seen that the content in the above method embodiment is applicable to the present storage medium embodiment, and the specific functions of the present storage medium embodiment are the same as those of the above method embodiment, and the achieved beneficial effects are the same as those of the above method embodiment.
The application uses the mobile phone APP as a user graphical interactive interface, and a user can perform operations such as user management, home management, authority setting and the like through the mobile phone APP. The user can set the use rights of different households for family members or strangers according to actual demands. Different speakers are distinguished by utilizing the voiceprint recognition technology, so that the purpose of judging the identity of the speaker is realized. And extracting instructions for controlling the home by the speaker by using a voice recognition technology. And integrating the instruction of the speaker for controlling the home, the speaker identity judged by voiceprint recognition and the control home authority set by the user, and judging whether the speaker has the authority for controlling the home instructed by the speaker. According to different authorities, the intelligent home terminal can make different reactions.
It should be appreciated that the layers, modules, units, and/or platforms, etc. included in the embodiment system of the application may be implemented or embodied by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer readable storage medium configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, in accordance with the methods and drawings described in the specific embodiments. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Furthermore, the data processing flows that the layers, modules, units, and/or platforms included in the system of embodiments of the present application correspond to perform may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The data processing flows that the layers, modules, units, and/or platforms included in the system of embodiments of the present application correspond to execute under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications), by hardware, or a combination thereof, that collectively execute on one or more processors. The computer program includes a plurality of instructions executable by one or more processors.
Further, the system may be implemented in any type of computing platform operatively connected to a suitable computing platform, including, but not limited to, a personal computer, mini-computer, mainframe, workstation, network or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and so forth. The data processing flows corresponding to the execution of the layers, modules, units, and/or platforms included in the system of the present application may be implemented in machine readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and/or write storage medium, RAM, ROM, etc., so that it may be read by a programmable computer, which when read by a computer, may be used to configure and operate the computer to perform the processes described herein. Further, the machine readable code, or portions thereof, may be transmitted over a wired or wireless network. When such media includes instructions or programs that, in conjunction with a microprocessor or other data processor, implement the steps described above, the application described herein includes these and other different types of non-transitory computer-readable storage media. The application also includes the computer itself when programmed according to the methods and techniques of the present application.
The present application is not limited to the above embodiments, but can be modified, equivalent, improved, etc. by the same means to achieve the technical effects of the present application, which are included in the spirit and principle of the present application. Various modifications and variations are possible in the technical solution and/or in the embodiments within the scope of the application.