CN111291801B - Data processing method and device - Google Patents

Abstract

The invention relates to the field of financial technology (Fintech), and discloses a data processing method and a data processing device, which are used for solving the problem that the prior art can not be trained to obtain a universal detection model, and the method comprises the following steps: the Federal server combines the data to be trained of each participant device, determines the public cutting characteristics at the public cutting point, constructs a detection model according to the public cutting characteristics at the public cutting point and sends the detection model to each participant device, so that each participant device can detect the abnormality of the data to be detected by using the detection model. The common cutting characteristics at the common cutting points are determined by combining the data to be trained of each participant device, so that the common cutting characteristics can simultaneously reflect the data characteristics of each participant device, the abnormal data in each participant device can be accurately detected by the detection model constructed based on the common cutting characteristics, the universality of the detection model is good, and the accuracy of abnormal detection is high.

Description

Data processing method and device

Technical Field

The invention relates to the technical field of financial technology (Fintech), in particular to a data processing method and device.

Background

With the development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually shifting to financial technology (Fintech), however, because the requirements of security and real-time performance of the financial industry are higher, higher requirements are also put forward on the technologies in the financial technology field. The abnormal data detection is a data processing method commonly used in the financial field, for example, when mass transaction data are processed, abnormal transaction data in the mass transaction data are detected by using a detection model, so that operation and maintenance personnel can maintain an abnormal transaction scene conveniently, and the stability of the financial field is improved.

At this stage, each participant device can usually train a detection model only by using locally stored data to be trained, however, the detection model trained by a single participant device cannot reflect the data characteristics of other participant devices. Therefore, the detection model trained by a single participant device can only accurately detect the abnormal data of the participant device, but cannot detect the abnormal data of other participant devices, or can misjudge the abnormal data of other participant devices, so that the detection accuracy of the abnormal data is low.

Disclosure of Invention

The invention provides a data processing method and a data processing device, which are used for training to obtain a universal detection model so as to realize detection of abnormal data of each participant device and further improve the accuracy of abnormal data detection.

In a first aspect, the present invention provides a data processing method applied to a federated server, where the method includes:

determining common cutting characteristics of each participant device at a common cutting point by combining to-be-trained data of each participant device; the public cutting feature is a feature for distinguishing abnormal data from normal data; further, according to the public cutting characteristics at the public cutting point, a detection model is constructed and sent to each participant device; and each participant device is also used for detecting data to be detected by using the detection model so as to determine whether the data to be detected is abnormal data.

In one possible implementation, the common cut feature comprises a common cut feature dimension and a common cut feature value; in a specific implementation, the determining, by combining to-be-trained data of each participant device, a common cutting feature of each participant device at a common cutting point includes: determining a common cutting feature dimension at the common cutting point according to the feature dimension of the data to be trained of each participant device at the common cutting point, and issuing the common cutting feature dimension at the common cutting point to each participant device, wherein the participant device is further configured to determine a cutting feature value of the participant device under the common cutting feature dimension based on the common cutting feature dimension at the common cutting point; correspondingly, the cutting characteristic value of each participant device under the public cutting characteristic dimension reported by each participant device is received, and the public cutting characteristic value is determined according to the cutting characteristic value of each participant device under the public cutting characteristic dimension.

In a possible implementation manner, the determining, according to the feature dimension of the data to be trained of the respective participant devices at the common cut point, the common feature dimension at the common cut point includes: and determining the common feature dimension of the data to be trained of each participant device at the common cutting point according to the feature dimension of the data to be trained of each participant device at the common cutting point, and selecting the common cutting feature dimension from the common feature dimensions.

In a possible implementation manner, the constructing a detection model according to the common cutting feature at the common cutting point includes: and associating any public cutting point with the public cutting characteristics at the public cutting points, connecting each public cutting point according to the inclusion relation of the data to be trained of each participant device at each public cutting point to obtain a binary tree model, and taking the binary tree model as the detection model.

In one possible implementation manner, the determining, by combining the data to be trained of each participant device, a common cutting feature of each participant device at a common cutting point includes: determining the common cutting characteristics of each participant device at the common cutting point corresponding to any training by combining the data to be trained of each participant device in any training, and constructing and obtaining a detection model corresponding to any training according to the common cutting characteristics at the common cutting point; correspondingly, the issuing the detection model to each participant device includes: and issuing the detection model corresponding to each training to each participant device, so that each participant device detects the data to be detected by using the detection model corresponding to each training to determine whether the data to be detected is abnormal data.

In a second aspect, the present invention provides a data processing method, applied to a participant device, where the method includes:

receiving a detection model sent by a federal server, and detecting data to be detected by using the detection model to determine whether the data to be detected is abnormal data; the detection model is obtained by combining the data to be trained of each participant device by the federal server, determining the public cutting characteristics of each participant device at a public cutting point and constructing according to the public cutting characteristics at the public cutting point; the common cut feature is used to distinguish abnormal data from normal data.

In a possible implementation manner, before receiving the detection model sent by the federal server, the public cutting feature dimension at the public cutting point sent by the federal server is also received; the public cutting feature dimension at the public cutting point is determined by the federal server according to the feature dimension of the data to be trained of each participant device at the public cutting point; further, determining a cutting feature value of the participant device under the public cutting feature dimension based on the public cutting feature dimension at the public cutting point, and reporting the cutting feature value of the participant device under the public cutting feature dimension to the federation server; the federal server is further used for determining the public cutting characteristic value according to the cutting characteristic value of each participant device under the public characteristic dimension.

In a possible implementation manner, the detecting, by using the detection model, data to be detected to determine whether the data to be detected is abnormal data includes: and cutting the data to be detected by using the public cutting characteristics at each public cutting point, determining the public cutting point to which the data to be detected is finally cut, if the weight corresponding to the public cutting point to be finally cut is greater than a first preset threshold value, determining that the data to be detected is abnormal data, and otherwise, determining that the data to be detected is normal data.

In a possible implementation manner, the receiving a detection model sent by a federated server includes: receiving a detection model corresponding to each training sent by the federal server; correspondingly, the detecting the data to be detected by using the detection model to determine whether the data to be detected is abnormal data includes: cutting data to be detected by using the common cutting features at each common cutting point in the detection model corresponding to any training, determining the common cutting point to which the data to be detected is finally cut in the detection model corresponding to any training, calculating the average weight corresponding to the common cutting point to which the data to be detected is finally cut in the detection model corresponding to each training, if the average weight is greater than a second preset threshold value, determining that the data to be detected is abnormal data, otherwise, determining that the data to be detected is normal data.

In a third aspect, the present invention provides a data processing apparatus, including:

the determining module is used for determining the common cutting characteristics of each participant device at a common cutting point by combining the data to be trained of each participant device; the public cutting feature is a feature for distinguishing abnormal data from normal data;

the construction module is used for constructing and obtaining a detection model according to the public cutting characteristics at the public cutting point;

the receiving and sending module is used for sending the detection model to each participant device; and each participant device is also used for detecting data to be detected by using the detection model so as to determine whether the data to be detected is abnormal data.

In one possible implementation, the common cut feature comprises a common cut feature dimension and a common cut feature value; the determining module is specifically configured to: determining a common cutting feature dimension at the common cutting point according to the feature dimension of the data to be trained of each participant device at the common cutting point, issuing the common cutting feature dimension at the common cutting point to each participant device, wherein the participant device is further configured to determine a cutting feature value of the participant device under the common cutting feature dimension based on the common cutting feature dimension at the common cutting point, receive the cutting feature value of each participant device under the common cutting feature dimension reported by each participant device, and determine the common cutting feature value according to the cutting feature value of each participant device under the common feature dimension.

In a possible implementation manner, the determining module is specifically configured to: and determining the common feature dimension of the data to be trained of each participant device at the common cutting point according to the feature dimension of the data to be trained of each participant device at the common cutting point, and selecting the common cutting feature dimension from the common feature dimensions.

In a possible implementation manner, the building module is specifically configured to: and associating any public cutting point with the public cutting characteristics at the public cutting points, connecting each public cutting point according to the inclusion relation of the data to be trained of each participant device at each public cutting point to obtain a binary tree model, and taking the binary tree model as the detection model.

In a possible implementation manner, the determining module is specifically configured to: determining the common cutting characteristics of each participant device at the common cutting point corresponding to any training by combining the data to be trained of each participant device in any training, and constructing and obtaining a detection model corresponding to any training according to the common cutting characteristics at the common cutting point; correspondingly, the transceiver module is specifically configured to: and issuing the detection model corresponding to each training to each participant device, so that each participant device detects the data to be detected by using the detection model corresponding to each training to determine whether the data to be detected is abnormal data.

In a fourth aspect, the present invention provides a data processing apparatus, including:

the receiving and sending module is used for receiving the detection model sent by the federal server; the detection model is obtained by combining the data to be trained of each participant device by the federal server, determining the public cutting characteristics of each participant device at a public cutting point and constructing according to the public cutting characteristics at the public cutting point; the common cutting feature is used for distinguishing abnormal data from normal data;

and the detection module is used for detecting the data to be detected by using the detection model so as to determine whether the data to be detected is abnormal data.

In a possible implementation manner, before the transceiver module receives the detection model sent by the federal server, the transceiver module also receives the public cutting feature dimension at the public cutting point issued by the federal server; the public cutting feature dimension at the public cutting point is determined by the federal server according to the feature dimension of the data to be trained of each participant device at the public cutting point; further, determining a cutting feature value of the participant device under the public cutting feature dimension based on the public cutting feature dimension at the public cutting point, and reporting the cutting feature value of the participant device under the public cutting feature dimension to the federation server; the federal server is further used for determining the public cutting characteristic value according to the cutting characteristic value of each participant device under the public characteristic dimension.

In a possible implementation manner, the detection module is specifically configured to: and cutting the data to be detected by using the public cutting characteristics at each public cutting point, determining the public cutting point to which the data to be detected is finally cut, if the weight corresponding to the public cutting point to be finally cut is greater than a first preset threshold value, determining that the data to be detected is abnormal data, and otherwise, determining that the data to be detected is normal data.

In a possible implementation manner, the transceiver module is specifically configured to: receiving a detection model corresponding to each training sent by the federal server; correspondingly, the detection module is specifically configured to: cutting data to be detected by using the common cutting features at each common cutting point in the detection model corresponding to any training, determining the common cutting point to which the data to be detected is finally cut in the detection model corresponding to any training, calculating the average weight corresponding to the common cutting point to which the data to be detected is finally cut in the detection model corresponding to each training, if the average weight is greater than a second preset threshold value, determining that the data to be detected is abnormal data, otherwise, determining that the data to be detected is normal data.

In a fifth aspect, the present invention provides a computing device, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, which, when executed by the processing unit, causes the processing unit to perform the method of any of the first or second aspects.

In a sixth aspect, the present invention provides a computer readable storage medium storing a computer program executable by a computing device, the program, when run on the computing device, causing the computing device to perform the method of any of the first or second aspects.

According to the data processing method and device provided by the invention, the public cutting characteristics at the public cutting point are determined by combining the data to be trained of each participant device, so that the public cutting characteristics can simultaneously reflect the data characteristics of each participant device, therefore, the detection model constructed based on the public cutting characteristics can accurately detect the abnormal data of each participant device, the universality of the detection model is better, and the detection accuracy is higher.

These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic diagram of a suitable system architecture according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of a data processing method according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a flow method for determining a common cutting feature according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of the distribution of data to be trained of participant devices at each common cut point;

fig. 5 is a schematic flowchart of a method for determining a next common cut point according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a detection model according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention;

FIG. 8 is a block diagram of another data processing apparatus according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a terminal device according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of a backend device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Fig. 1 is a schematic diagram of a suitable system architecture according to an embodiment of the present invention, and as shown in fig. 1, the system architecture may include afederal server 110 and at least two participant devices, such as aparticipant device 121, aparticipant device 122, and aparticipant device 123. Thefederal server 110 may be connected to each participant device, for example, through a wired connection or a wireless connection, which is not limited in particular.

Based on the system architecture illustrated in fig. 1, fig. 2 is an interaction flow diagram corresponding to a data processing method provided in an embodiment of the present invention, where the method includes:

step 201, the federal server combines the data to be trained of each participant device to determine the common cutting characteristics of each participant device at the common cutting point.

In the embodiment of the invention, the common cutting point is a unified cutting node determined by combining each participant device when cutting the data to be trained of each participant device, each participant device corresponds to the data to be trained at the common cutting point, and the federal server can combine the data to be trained of each participant device at the common cutting point to determine the common cutting characteristics at the common cutting point and then send the common cutting characteristics to each participant device. Correspondingly, any participant device cuts the data to be trained of the participant device at the common cutting point by using the common cutting features at the common cutting point to obtain a subsequent common cutting point, namely, the data to be trained of the participant device at the common cutting point is cut into the subsequent common cutting point, and the above processes are executed in a circulating manner until the data to be trained of the participant device at the common cutting point is cut to be incapable of being cut, and each common cutting point and the common cutting features at each common cutting point are correspondingly obtained.

Wherein the common cutting feature is a feature for distinguishing abnormal data from normal data. The normal data and the abnormal data are relative to each data to be trained, the normal data refers to data with characteristics similar to most of the data to be trained, and the abnormal data refers to data with characteristics different from most of the data to be trained.

And 202, the federal server constructs a detection model according to the public cutting characteristics at the public cutting point.

And step 203, the federal server issues the detection model to each participant device.

And 204, detecting the data to be detected by each participant device by using the detection model so as to determine whether the data to be detected is abnormal data.

In the embodiment of the invention, the public cutting characteristics at the public cutting points are determined by combining the data to be trained of each participant device, so that the public cutting characteristics can simultaneously reflect the data characteristics of each participant device, thus, the detection model constructed based on the public cutting characteristics can accurately detect the abnormal data of each participant device, the universality of the detection model is better, and the detection accuracy is higher.

In step 201, the federal server can determine the common cutting characteristics of the various participant devices at any common cutting point in the manner illustrated in fig. 3. Wherein the common cut feature may comprise a common cut feature dimension and a common cut feature value.

Fig. 3 is a schematic diagram of a flow method for determining a common cutting feature at any common cutting point according to an embodiment of the present invention, as shown in fig. 3, the method includes:

step 301, determining a common cutting feature dimension at a common cutting point according to the feature dimension of the data to be trained of each participant device at the common cutting point.

In one example, for any common cut point, thefederal server 110 may determine the common feature dimension of the data to be trained of each participant device at the common cut point according to the feature dimension of the data to be trained of each participant device at the common cut point, and then select the common cut feature dimension at the common cut point from the common feature dimensions. For example, thefederal server 110 may send a dimension acquisition request to each participant device, where the dimension acquisition request carries an identifier of the common cut point, so that each participant device may determine and report the feature dimension of the data to be trained of each participant device at the common cut point according to the dimension acquisition request, or each participant device may report the feature dimension of the data to be trained of each participant device at the common cut point according to a fixed period or in real time, without limitation.

In one possible approach, the common cut feature dimensions at any two common cut points may be different. In a specific implementation, before performing model training, thefederal server 110 may first obtain the feature dimension of the data to be trained in each participant device, and then construct a common feature dimension set using the common feature dimension of the data to be trained in each participant device. In this way, for any common cutting point, thefederation server 110 may determine, from the set of common feature dimensions, each common feature dimension different from the common cutting feature dimensions at other common cutting points, and then select one of the common feature dimensions as the common cutting feature dimension of each participant device at the common cutting point.

In this approach, the common feature dimension set may have several possible scenarios as follows:

in case one, only the common feature dimensions different from the common cutting feature dimensions at other common cutting points are stored in the common feature dimension set.

In a specific implementation, for any common cut point, thefederation server 110 may randomly select a common feature dimension directly from the set of common feature dimensions and use the common feature dimension as the common cut feature dimension of each participant at the common cut point. Accordingly, after the common cut point cutting is finished, thefederation server 110 may delete the common cut feature dimension of each participant at the common cut point from the common feature dimension set, so that only the common feature dimension different from the common cut feature dimension of each participant at other common cut points is stored in the common feature dimension set.

In case two, all public feature dimensions and states of all public feature dimensions are stored in the public feature dimension set; the state of any common feature dimension is a selected state or an unselected state, the selected state is used for indicating that the common feature dimension is a common cutting feature dimension at other common cutting points, and the unselected state is used for indicating that the common feature dimension is not a common cutting feature dimension at other common cutting points.

In a specific implementation, for any common cutting point, thefederal server 110 may determine a state of each common feature dimension from the common feature dimension set, randomly select one common feature dimension from the common feature dimensions in which the states are unselected, and use the common feature dimension as the common cutting feature dimension at the common cutting point. Correspondingly, after the cutting of the common cutting point is finished, thefederal server 110 may update the state of the common cutting feature dimension at the common cutting point in the common feature dimension set to a selected state, so as to update the state of each common feature dimension in the common feature dimension set in real time, and ensure the accuracy of determining the common cutting feature dimension.

In the mode, the public feature dimension different from the public cutting feature dimension at other public cutting points is selected as the public cutting feature dimension at the public cutting points, so that each public cutting point can be cut by using different feature dimensions, the data feature of each feature dimension can be used for cutting the data to be trained more uniformly, and the accuracy of the public cutting feature at the public cutting points is improved.

And 302, issuing the public cutting feature dimension at the public cutting point to each participant device.

Step 303, the participant device determines a cutting feature value of the participant device under the common cutting feature dimension based on the common cutting feature dimension at the common cutting point.

Here, the common cut point may be any one of the 1 st to nth common cut layers, the common cut point in the 1 st common cut layer is a root common cut point, and the data to be trained of the participant device at the root common cut point includes all the data to be trained of the participant device in model training. Accordingly, the common cutting point in the ith (0< i < N-2, and is an integer) level common cutting layer is an intermediate common cutting point or a leaf common cutting point, any intermediate common cutting point in the ith level common cutting layer connects at least one common cutting point in the (i + 1) th level common cutting layer, the data to be trained of the participant device at any intermediate common cutting point in the ith level common cutting layer includes the data to be trained of the participant device at all common cutting points in the (i + 1) th level common cutting layer to which the intermediate common cutting point is connected, and the leaf common cutting point in the ith level common cutting layer does not connect the common cutting points in any of the (i + 1) th level common cutting layers.

For example, fig. 4 is a schematic distribution diagram of data to be trained of the participant devices at each common cutting point, as shown in fig. 4, a root common cutting point a is arranged in the level 1 common cutting layer₁Root common cutting Point A₁Including all data to be trained in the participant device, i.e. data to be trained a₁Data to be trained a₂Data to be trained a₃Data to be trained a₄Data to be trained a₅Data to be trained a₆And data a to be trained₇. Accordingly, the root common cut point A₁Connecting intermediate common cut points A in level 2 common cut layers₂₁And leaf common cutting point A₂₂With participant devices at an intermediate common cut point A₂₁The data to be trained of (A) includes data to be trained (a)₁Data to be trained a₃Data to be trained a₄Data to be trained a₅Data to be trained a₆And data a to be trained₇With participant devices at the leaf common cut point A₂₂The data to be trained of (A) includes data to be trained (a)₂. And, the middle common cutting point A₂₁Connecting leaf common cutting points A in a 3 rd level common cutting layer₃₁And leaf common cutting point A₃₂With participant devices at the leaf common cut point A₃₁The data to be trained of (A) includes data to be trained (a)₁Data to be trained a₄And data a to be trained₇With participant devices at the leaf common cut point A₃₂The data to be trained of (A) includes data to be trained (a)₃Data to be trained a₅And data a to be trained₆。

In specific implementation, for any common cutting point, the participant device may first obtain data to be trained of the participant device in the common cutting point, and then determine, according to a common cutting feature dimension of the common cutting point, each feature value of the data to be trained in the common cutting point under the common cutting feature dimension, and if the common cutting feature dimension corresponds to a feature dimension with discrete feature values, the participant device may randomly select one feature value from each feature value of the data to be trained under the common cutting feature dimension as a cutting feature value of the participant device at the common cutting point. Accordingly, if the common cutting feature dimension corresponds to a feature dimension with continuous feature values, the participant device may randomly select an intermediate feature value from the maximum feature value and the minimum feature value of the data to be trained in the common cutting feature dimension as the cutting feature value of the participant device at the common cutting point. The manner of selecting the intermediate feature value may be set by a person skilled in the art based on experience, for example, the intermediate feature value may be randomly selected, or an average feature value of the maximum feature value or the minimum feature value may also be used as the intermediate feature value, or a weighted average feature value of the maximum feature value or the minimum feature value may also be used as the intermediate feature value, which is not limited specifically.

For example, table 1 is a schematic table of data to be trained of participant devices at a common cut point.

TABLE 1

As shown in table 1, the participant devices are at common cut point a₂₁The data to be trained of (A) includes data to be trained (a)₁Data to be trained a₃Data to be trained a₄Data to be trained a₅Data to be trained a₆And data a to be trained₇With participant devices at common cut point A₂₁The characteristic dimensions of (a) include the amount of consumption, time of purchase, age, and shopping category.

In specific implementation, if the public cutting feature dimension is the consumption amount, the participant device may first query the table 1 to determine the public cutting point a₂₁The respective characteristic value of the data to be trained under the consumption amount, namely 210, 600, 53, 1000, 860,100. Further, since the amount of consumption belongs to a feature dimension with continuous feature values, the participant device may determine the common cut point a first₂₁The maximum consumption amount of each data to be trained is 1000, the minimum consumption amount is 53, and the sum is [53, 1000 ]]In the method, a consumption amount is randomly selected as a participant device at a common cutting point A₂₁Such as 520.

Alternatively, if the public cut feature dimension is a shopping category, the participant device may first query table 1 to determine a public cut point a₂₁And (4) processing various characteristic values of the data to be trained under the shopping category, namely fan heaters, furniture, snacks, game machines, washing machines and clothes. Further, since the shopping category belongs to a feature dimension with discrete feature values, the participant device may randomly select one feature value from the respective feature values as the participant device at the common cut point a₂₁Such as a gaming machine.

And step 304, the participant equipment reports the cutting characteristic value of the participant equipment under the public cutting characteristic dimension to a federal server.

And 305, determining a common cutting characteristic value according to the cutting characteristic values of the participator devices under the common cutting characteristic dimension.

In specific implementation, after receiving the cutting feature values of the respective participant devices in the common cutting feature dimension, thefederal server 110 may determine the common cutting feature value in multiple ways, for example, may randomly select one cutting feature value as the common cutting feature value, or may also use an average cutting feature value of the respective cutting feature values as the common cutting feature value, or may also use a weighted average cutting feature value of the respective cutting feature values as the common cutting feature value, and the like, without limitation.

And step 306, taking the public cutting feature dimension and the public cutting feature value as the public cutting feature at the public cutting point.

In the embodiment of the invention, the participant equipment can report the cutting characteristic value to the federal server only without reporting the data to be trained, so that the safety of the data to be trained in the participant equipment can be protected, the data transmission quantity is reduced, and the training efficiency is improved. In addition, the cutting characteristic value is generated based on the characteristic value of the data to be trained in the participant equipment under the public cutting characteristic dimension, so that the cutting characteristic value can accurately reflect the data characteristic of the data to be trained in the participant equipment.

The above describes the implementation process of determining the common cutting feature dimension and the common cutting feature value at any common cutting point, and how to determine the next common cutting point is described below.

Fig. 5 is a flowchart of a method for determining a next common cutting point according to an embodiment of the present invention, as shown in fig. 5, the method includes:

and step 501, the federal server issues the public cutting characteristic value at the public cutting point to each participant device.

In a specific implementation, thefederal server 110 may directly issue the public cutting feature value at the public cutting point to each participant device, or may first encrypt the public cutting feature value at the public cutting point and then issue the encrypted public cutting feature value to each participant device, so as to ensure the security of data transmission, which is not specifically limited.

And 502, the participant device cuts the data to be trained of the participant device at the common cutting point by using the common cutting characteristic value at the common cutting point to obtain a cutting result.

In specific implementation, if the number of the data to be trained of the participant device at the common cutting point is less than or equal to 1, the data to be trained of the participant device at the common cutting point cannot be cut, and therefore it can be determined that the cutting result is a cutting failure. Correspondingly, if the number of the data to be trained of the participant equipment at the common cutting point is greater than 1, the data to be trained of the participant equipment at the common cutting point can be directly cut by using the common cutting feature value, and after the cutting is completed, the cutting result is determined to be successful.

For example, based on the data to be trained as shown in table 1, when the common cut point is the common cut point a₂₁When the public cutting feature dimension is an expense amount 500, as shown in fig. 4, the participant device may use the expense amount 500 to determine that the participant device is at a public cutting point a₂₁To-be-trained data a of₁Data to be trained a₃Data to be trained a₄Data to be trained a₅Data to be trained a₆And data a to be trained₇And (6) cutting. Due to the data a to be trained₁(amount of consumption 210) and data to be trained₄(amount of consumption 53) and data to be trained a₇The spending amount is less than 500, so the data a to be trained can be processed₁Data to be trained a₄And data a to be trained₇Common cutting point A divided into 3 rd common cutting layer₃₁In (1). Accordingly, due to the data a to be trained₃(the consumption amount is 600) and data a to be trained₅(the amount of consumption is 1000) and data a to be trained₆The spending amount (the spending amount is 860) is more than or equal to 500, so that the data a to be trained can be processed₃Data to be trained a₅And data a to be trained₆Common cutting point A divided into 3 rd common cutting layer₃₂In (1). And after the cutting is finished, the participant equipment determines that the cutting result is successful.

Or, when the common cutting point is the common cutting point A₂₂When, as shown in fig. 4, since the participant devices are at the common cut point a₂₂The data to be trained of (A) includes only the data to be trained (a)₂Thus, it may be determined that the participant device is at the common cut point a₂₂The data to be trained in (1) cannot be cut, and the participant equipment determines that the cutting result is cutting failure.

It should be noted that the above is only an exemplary description, and does not limit the present invention, and in the specific implementation, the dividing manner may be set by those skilled in the art according to experience, for example, the dividing manner may also be set by those skilled in the artTo divide the data to be trained with the consumption sum of more than or equal to 500 into common cutting points A₃₁In the method, the data to be trained with the consumption sum less than 500 is divided into common cutting points A₃₂In (1).

And step 503, the participant equipment reports the cutting result to the federal server.

And step 504, the federal server determines whether the cutting result of each participant device meets the end condition of model training, if not, step 505 is executed, and if so, step 506 is executed.

In the embodiment of the present invention, the ending condition of the model training may be any one or more of the following: the depth of the common cutting points (the distance between the common cutting point and the root common cutting point) is greater than or equal to a preset cutting depth, the common cutting points which are not cut and can be cut do not exist in each participant device, the number of times of cutting is greater than or equal to the preset cutting number, the time length of cutting is greater than or equal to the preset cutting time length, and the number of the common cutting points included in the highest-level common cutting layer is greater than or equal to the preset number. By setting the ending conditions, the data processing method in the embodiment of the invention has a wider application range and can better meet the requirements of users.

And 505, the federal server combines the data to be trained of each participant device in each common cutting point to determine the next common cutting point.

For ease of understanding, the following describes a specific implementation process of determining step 505 and step 506 by taking the end condition of model training as an example including the above items:

step a, after receiving cutting results sent by each participant device, the federal server firstly judges whether the number of times of cutting is executed is greater than or equal to a preset cutting number, and/or judges whether the time length of cutting is executed is greater than or equal to a preset cutting time length, and/or judges whether the number of public cutting points included in the highest-level public cutting layer is greater than or equal to a preset number, if at least one item is yes, the cutting results can be determined to meet the end conditions of model training, step b is executed, and if all the items are not, step c is executed.

And b, the federal server determines that the next common cutting point does not exist.

And c, the federal server determines whether each participant device is successfully cut according to the cutting result of each participant device, if it is determined that each participant device cannot be successfully cut, the common cutting point is a leaf common cutting point, and the common cutting point cannot be cut downwards any more, and the step e is executed. And if the fact that the cutting of some participant equipment is successful is determined, judging whether the depth of the public cutting point obtained by cutting is larger than or equal to the preset depth, if not, executing the step d, and if so, executing the step e.

And d, the federal server takes the left public cutting point in the next public cutting layer connected with the public cutting point as the next public cutting point.

In the embodiment of the invention, if the federal server determines that one or more participant devices are successfully cut and the current branch does not reach the set cutting depth, the cutting work of the current branch can be continued, namely, a left common cutting point in a next-stage common cutting layer connected with the common cutting point on the current branch is used as a next common cutting point. By automatically executing the next cutting when the cutting work of the current branch is not finished, the left common cutting point of the current branch can be cut to the point where the cutting cannot be performed or the preset depth is reached, so that the cutting continuity is ensured, and the data processing efficiency is improved.

And e, the federal server issues a query instruction to each participant device.

And f, the participant equipment determines whether an uncut and cuttable common cutting point exists in the participant equipment according to the query instruction, if so, the uncut and cuttable common cutting point with the deepest cutting depth is used as the next common cutting point cuttable in the participant equipment, and if not, the participant equipment is determined not to have the next common cutting point.

Here, the uncut and cuttable common cutting point refers to a common cutting point where the data to be trained of the participant device in the common cutting point is greater than 1, and the depth of the common cutting point is less than the preset cutting depth.

In a specific implementation, after each participant device receives a query instruction, whether an uncut and cuttable common cutting point exists in the participant device can be queried and determined, if so, a common cutting point with the deepest depth can be obtained from all uncut and cuttable common cutting points, and a query result is generated according to the hierarchical relationship of the common cutting points; the hierarchical relationship of the common cutting point may include a common cutting layer where the common cutting point is located and a position of the common cutting point in the common cutting layer. Accordingly, if not, the query result may be generated according to the indication message that the next common cut point does not exist in the participant device.

Step g, each participant device reports the query result to the federal server; and the query result is the hierarchical relationship of the next common cut point which can be cut in the participant equipment, or is an indication message of the absence of the next common cut point in the participant equipment.

Step h, the federal server determines that the end condition of model training is met if determining that the next common cutting point capable of being cut does not exist in all the participant devices according to the query result reported by each participant device, and executes step b; if it is determined that there is a next common cut point in the one or more participant devices that is cuttable, step i is performed.

And step i, the federal server selects the public cutting point with the deepest hierarchy and the nearest position as the next public cutting point of each participant device according to the hierarchy relation of each next public cutting point reported by one or more participant devices.

The common cutting point with the deepest level and the nearest position refers to the common cutting point which is deepest in the common cutting layer and is closest to the position of the cut common cutting point in the common cutting layer.

In the embodiment of the invention, when the common cutting points on the initial branch are cut to the preset cutting depth or cannot be cut, if the federal server inquires that the uncut and cuttable common cutting points exist in each participant device, the common cutting point with the deepest depth can be selected from the uncut and cuttable common cutting points of each participant device to be used as the next common cutting point, and the cutting is executed in a circulating manner until the uncut and cuttable common cutting points do not exist in each participant device. Therefore, the cutting depth is used as a reference to cut each public cutting point from deep to shallow, so that the ordered cutting can be guaranteed, the public cutting points are prevented from being omitted, and the accuracy of data processing and the detection effect of the detection model are improved.

And step 506, the federal server determines that the next common cutting point does not exist, and a detection model is constructed according to the common cutting characteristics of all the common cutting points.

In one possible implementation, the federated server may construct the detection model in the following manner: and associating any one common cutting point with the common cutting features of each participant device at the common cutting point, connecting each common cutting point according to the inclusion relation of the to-be-trained data of each participant device at each common cutting point to obtain a binary tree model, and taking the binary tree model as a detection model.

Fig. 6 is a schematic structural diagram of a detection model provided in an embodiment of the present invention, and as shown in fig. 6, when a preset cutting depth is 4, the first cutting cuts data to be trained of any participant device at a common cutting point 1 using a common cutting feature dimension 1 and a common cutting feature value 1, and cuts the data to be trained of any participant device at the common cutting point 1 into a common cutting point 2 and a common cutting point 3. Since the cutting depth at this time is 2, and the preset cutting depth is not reached, the left common cutting point (i.e., the common cutting point 2) in the next common cutting layer can be used as the next common cutting point. Based on this, the second cutting cuts the data to be trained of any participant device at the common cutting point 2 by using the common cutting characteristic dimension 2 and the common cutting characteristic value 2, and cuts the data to be trained of any participant device at the common cutting point 2 into the common cutting 4 and the common cutting point 5.

Accordingly, the cutting depth at this time is 3, and the preset cutting depth is not reached, so that the left common cutting point (i.e., the common cutting point 4) in the next common cutting layer of the common cutting point 2 can be used as the next common cutting point. Based on this, the third cutting cuts the data to be trained of any participant device at the common cutting point 4 by using the common cutting characteristic dimension 3 and the common cutting characteristic value 3, and cuts the data to be trained of any participant device at the common cutting point 4 into the left sample space and the right sample space of the common cutting point 4. Since the cutting depth at this time is 4, and the preset cutting depth is reached, the federal server determines that the current branch can not be cut.

Further, the federal server sends a query instruction to each participant device, and determines the common cutting point with the deepest depth and capable of being cut as the common cutting point 5 according to a query result returned by each participant device, so that the common cutting point 5 is used as the next common cutting point. Based on this, the fourth cutting cuts the data to be trained of any participant device at the common cutting point 5 by using the common cutting feature dimension 4 and the common cutting feature value 4, and cuts the data to be trained of any participant device at the common cutting point 5 into the left sample space and the right sample space of the common cutting point 5. Since the cutting depth at this time is 4, and the preset cutting depth is reached, the federal server determines that the current branch can not be cut.

Correspondingly, the federal server resends the query instruction to each participant device, and determines the common cutting point with the deepest depth and capable of being cut as the common cutting point 3 according to the query result returned by each participant device, so that the common cutting point 3 is used as the next common cutting point. Based on this, the fifth cutting cuts the data to be trained of any participant device at the common cutting point 3 by using the common cutting feature dimension 5 and the common cutting feature value 5, and cuts the data to be trained of any participant device at the common cutting point 3 into the left sample space and the right sample space of the common cutting point 3. Since the cutting depth at this time is 4, and the preset cutting depth is reached, the federal server determines that the current branch can not be cut.

In this way, the federal server continues to send query instructions to each participant device, and determines that each participant device does not store a cuttable common cutting point according to a query result returned by each participant device, so that the federal server determines that a model training end condition is met.

After the cutting is determined to be completed, the federal server may associate each common cutting point with the common cutting feature of each participant device at the common cutting point, and then connect each common cutting point according to the inclusion relationship of the to-be-trained data of each participant device at each common cutting point to obtain the binary tree model illustrated in fig. 6, i.e., the detection model.

In the embodiment of the invention, the federal server determines the common cutting characteristics at the common cutting point by combining each participant device, and each participant device also cuts the data to be trained of each participant device at the common cutting point by using the common cutting characteristics at the common cutting point, so that the federal server and each participant device actually realize the synchronous operation effect of training and detecting at the same time, when a detection model is obtained by training, the data to be trained in each participant device is also divided into different common cutting points, and the abnormality of the data to be trained in each participant device is determined. Obviously, the embodiment of the invention can realize multiple times of model application through one-time model training, and synchronously realize the detection of the data to be trained in each participant device in the process of obtaining one detection model through training, so that the efficiency of model detection is higher.

In the embodiment of the present invention, thefederal server 110 may only perform 1 model training in combination with each participant to obtain 1 detection model, or may perform multiple model training in combination with each participant device to obtain multiple detection models, if only 1 detection model is obtained by training, each participant may use all respective data to be trained as data to be trained used by the model training, and if multiple detection models are obtained by training, each participant device may select part of data to be trained from all respective data to be trained before each model training as data to be trained used by each model training. The number of the data to be trained selected by each participant device for each model training can be the same or different, and the data to be trained used by the same participant device in each model training can be not completely the same, so that the detection model can be ensured to assemble the data characteristics of different data to be trained, and the detection effect of the detection model is improved.

In a possible implementation manner, before performing model training, thefederal server 110 may issue a sample confirmation instruction to each participant device, and after any participant device receives the sample confirmation instruction, if it is determined that the number of all data to be trained of the participant device is less than or equal to a preset number, all data to be trained in the participant device may be used as the data to be trained of the current model training, and all data to be trained may be used as the data to be trained in the root common cut point. Correspondingly, if the number of all the data to be trained in the participant device is greater than the preset number, part of the data to be trained can be selected from all the data to be trained as the data to be trained in the model training, and the selected part of the data to be trained is used as the data to be trained in the root common cut point. In this kind of implementation, through treat that training data is less when treating that training data will all treat training data as the data of treating training of model training, and treat training data and select the part when more and treat training data as the data of treating training of model training, can reduce the data bulk of model training at every turn when fully guaranteeing the sample variety of treating training data, when improving data processing's efficiency, improve the accuracy of detection model.

Based on this, if only one detection model is obtained by training, in step 204, the participant device may detect the abnormality of the data to be detected by the following method:

for any data to be detected, the participant equipment can firstly use the public cutting features at each public cutting point to cut the data to be detected, determine the public cutting point to which the data to be detected is finally cut, if the weight corresponding to the public cutting point to be finally cut is greater than a first preset threshold value, determine that the data to be detected is abnormal data, otherwise, determine that the data to be detected is normal data.

The first preset threshold may be set by a person skilled in the art based on experience, or may be set according to actual needs, and is not limited specifically. In one example, the first preset threshold may be set to 0.5.

For example, referring to the detection model illustrated in fig. 6, the data to be detected may be cut by using the public cutting characteristic dimension 1 and the public cutting characteristic value 1 at the public cutting point 1, if the characteristic value of the data to be detected in the public cutting characteristic dimension 1 is less than or equal to the public cutting characteristic value 1, the data to be detected may be cut into the public cutting point 2, and if the characteristic value of the data to be detected in the public cutting characteristic dimension 1 is greater than the public cutting characteristic value 1, the data to be detected may be cut into the public cutting point 3. Taking the example that the data to be detected is cut into the common cutting point 2, the participant device may cut the data to be detected by using the common cutting characteristic dimension 2 and the common cutting characteristic value 2 at the common cutting point 2, if the characteristic value of the data to be detected under the common cutting characteristic dimension 2 is less than or equal to the common cutting characteristic value 2, the data to be detected may be cut into the common cutting point 4, and if the characteristic value of the data to be detected under the common cutting characteristic dimension 2 is greater than the common cutting characteristic value 2, the data to be detected may be cut into the common cutting point 5. And continuously executing the process until the data to be detected is cut to be uncut.

Further, after the cutting is completed, thefederal server 110 may first determine the public cutting point to which the data to be detected is cut last, and then obtain the weight of the public cutting point to which the data is cut last. The longer the distance between the public cutting point to which the data to be detected is finally cut and the root public cutting point is, the more similar the data to be detected and most of the data to be trained are, the lighter the abnormal degree of the data to be detected is, the closer the distance between the public cutting point to which the data to be detected is finally cut and the root public cutting point is, the larger the difference between the data to be detected and most of the data to be trained is, and the more serious the abnormal degree of the data to be detected is. Based on this, after obtaining the weight of the last cut public cutting point, thefederal server 110 may determine whether the weight is greater than a first preset threshold, if so, determine that the data to be detected is abnormal data, and if not, determine that the data to be detected is normal data.

Since the distances between each common cutting point and the root common cutting point of the same-level common cutting layer are the same, the same weight may be set for each common cutting point of the same-level common cutting layer, for example, the weight 0.8 is set for the common cutting point 2 and the common cutting point 3 of the second-level common cutting layer in fig. 6, and the weight 0.3 is set for the common cutting point 4 and the common cutting point 5 of the third-level common cutting layer. Thus, when the first preset threshold is set to 0.5, if the data to be detected is finally cut into the right sample space of the common cut point 5, it is determined that the last cut common cut point is the common cut point 5, and since the weight of the common cut point 5 is 0.3 (less than 0.5), the data to be detected is normal data; correspondingly, if the data to be detected is cut into the left sample space of the common cut point 3 at last, it is determined that the common cut point cut at last is the common cut point 3, and the data to be detected is abnormal data because the weight of the common cut point 3 is 0.8 (greater than 0.5).

Accordingly, if a plurality of detection models are trained, in step 204, the participant device may detect an abnormality of the data to be detected by:

for any data to be detected, the participant equipment may cut the data to be detected by using the common cutting features at the common cutting points in the detection model corresponding to any training, determine the common cutting points to which the data to be detected is finally cut in the detection model corresponding to any training, then calculate the average weight corresponding to the common cutting points to which the data to be detected is finally cut in the detection model corresponding to each training, determine that the data to be detected is abnormal data if the average weight is greater than a second preset threshold, and determine that the data to be detected is normal data otherwise.

The second preset threshold may be set by a person skilled in the art based on experience, or may be set according to actual needs, and is not limited specifically. In one example, the second preset threshold may be set to 0.5.

It should be noted that the foregoing is only an exemplary illustration, and does not constitute a limitation to the present solution, and in a specific implementation, the participant device may also calculate a weighted average weight corresponding to the finally cut common cut point, and determine the abnormality of the data to be detected by comparing the weighted average weight with a second preset threshold. The weighted weight value may be determined based on loss functions of a plurality of detection models, and if the loss function of a detection model is smaller, it indicates that the detection effect is better, a larger weight value may be set for the detection model, and if the loss function of a detection model is larger, it indicates that the detection effect is worse, and a smaller weight value may be set for the detection model.

In the embodiment of the invention, the federal server combines the data to be trained of each participant device to determine the public cutting characteristics of each participant device at the public cutting point; the public cutting feature is a feature for distinguishing abnormal data from normal data; therefore, the federal server constructs a detection model according to the public cutting characteristics at the public cutting point, and issues the detection model to each participant device, so that each participant device can detect data to be detected by using the detection model, and determine whether the data to be detected is abnormal data. In the embodiment of the invention, the public cutting characteristics at the public cutting points are determined by combining the data to be trained of each participant device, so that the public cutting characteristics can simultaneously reflect the data characteristics of each participant device, thus, the detection model constructed based on the public cutting characteristics can accurately detect the abnormal data of each participant device, the universality of the detection model is better, and the accuracy of the abnormal detection is higher.

In view of the above method flow, an embodiment of the present invention further provides a data processing apparatus, and specific contents of the apparatus may be implemented with reference to the above method.

Fig. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention, where the apparatus includes:

a determiningmodule 701, configured to determine, in combination with data to be trained of each participant device, a common cutting feature of each participant device at a common cutting point; the public cutting feature is a feature for distinguishing abnormal data from normal data;

abuilding module 702, configured to build a detection model according to the common cutting feature at the common cutting point;

atransceiver module 703, configured to issue the detection model to each of the participant devices; and each participant device is also used for detecting data to be detected by using the detection model so as to determine whether the data to be detected is abnormal data.

Optionally, the common cutting feature comprises a common cutting feature dimension and a common cutting feature value;

the determiningmodule 701 is specifically configured to:

determining a common cutting feature dimension at the common cutting point according to the feature dimension of the data to be trained of each participant device at the common cutting point;

issuing the public cutting feature dimension at the public cutting point to each participant device, wherein the participant device is further configured to determine a cutting feature value of the participant device under the public cutting feature dimension based on the public cutting feature dimension at the public cutting point;

and receiving the cutting characteristic values of the participant devices under the public cutting characteristic dimension reported by the participant devices, and determining the public cutting characteristic values according to the cutting characteristic values of the participant devices under the public cutting characteristic dimension.

Optionally, the determiningmodule 701 is specifically configured to:

and determining the common feature dimension of the data to be trained of each participant device at the common cutting point according to the feature dimension of the data to be trained of each participant device at the common cutting point, and selecting the common cutting feature dimension from the common feature dimensions.

Optionally, thebuilding module 702 is specifically configured to:

associating any common cutting point with a common cutting feature at the common cutting point;

and connecting the public cutting points according to the inclusion relation of the data to be trained of each participant device at each public cutting point to obtain a binary tree model, and taking the binary tree model as the detection model.

Optionally, the determiningmodule 701 is specifically configured to:

determining the common cutting characteristics of each participant device at the common cutting point corresponding to any training by combining the data to be trained of each participant device in any training, and constructing and obtaining a detection model corresponding to any training according to the common cutting characteristics at the common cutting point;

thetransceiver module 703 is specifically configured to:

and issuing the detection model corresponding to each training to each participant device, so that each participant device detects the data to be detected by using the detection model corresponding to each training to determine whether the data to be detected is abnormal data.

Fig. 8 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present invention, where the apparatus includes:

the receiving and sendingmodule 801 is used for receiving the detection model sent by the federal server; the detection model is obtained by combining the data to be trained of each participant device by the federal server, determining the public cutting characteristics of each participant device at a public cutting point and constructing according to the public cutting characteristics at the public cutting point; the common cutting feature is used for distinguishing abnormal data from normal data;

the detectingmodule 802 is configured to detect data to be detected by using the detection model, so as to determine whether the data to be detected is abnormal data.

Optionally, before thetransceiver module 801 receives the detection model sent by the federal server, the transceiver module is further configured to:

receiving public cutting feature dimensions at a public cutting point issued by a federal server; the public cutting feature dimension at the public cutting point is determined by the federal server according to the feature dimension of the data to be trained of each participant device at the public cutting point;

determining a cut feature value of the participant device at the common cut feature dimension based on the common cut feature dimension at the common cut point;

reporting the cutting characteristic value of the participant equipment under the public cutting characteristic dimension to the federal server; the federal server is further used for determining the public cutting characteristic value according to the cutting characteristic value of each participant device under the public characteristic dimension.

Optionally, thedetection module 802 is specifically configured to:

cutting the data to be detected by using the public cutting characteristics at each public cutting point, and determining the public cutting point to which the data to be detected is finally cut;

and if the weight corresponding to the finally cut public cutting point is greater than a first preset threshold value, determining that the data to be detected is abnormal data, otherwise, determining that the data to be detected is normal data.

Optionally, thetransceiver module 801 is specifically configured to:

receiving a detection model corresponding to each training sent by the federal server;

thedetection module 802 is specifically configured to:

cutting the data to be detected by using the common cutting features at each common cutting point in the detection model corresponding to any one training, and determining the common cutting point to which the data to be detected is finally cut in the detection model corresponding to any one training;

and calculating the average weight corresponding to the common cutting point to which the data to be detected is finally cut in the detection model corresponding to each training, if the average weight is greater than a second preset threshold value, determining that the data to be detected is abnormal data, and otherwise, determining that the data to be detected is normal data.

From the above, it can be seen that: in the embodiment of the invention, the federal server combines the data to be trained of each participant device to determine the public cutting characteristics of each participant device at the public cutting point; the public cutting feature is a feature for distinguishing abnormal data from normal data; therefore, the federal server constructs a detection model according to the public cutting characteristics at the public cutting point, and issues the detection model to each participant device, so that each participant device can detect data to be detected by using the detection model, and determine whether the data to be detected is abnormal data. In the embodiment of the invention, the public cutting characteristics at the public cutting points are determined by combining the data to be trained of each participant device, so that the public cutting characteristics can simultaneously reflect the data characteristics of each participant device, thus, the detection model constructed based on the public cutting characteristics can accurately detect the abnormal data of each participant device, the universality of the detection model is better, and the accuracy of the abnormal detection is higher.

Based on the same inventive concept, an embodiment of the present invention provides a computing device, including at least one processing unit and at least one storage unit, where the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit is caused to execute the method described in any of fig. 2 to 5.

Based on the same inventive concept, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program executable by a computing device, and when the program runs on the computing device, the computer program causes the computing device to execute the method described in any of fig. 2 to 5.

Based on the same technical concept, an embodiment of the present invention provides a terminal device, as shown in fig. 9, including at least oneprocessor 901 and amemory 902 connected to the at least one processor, where a specific connection medium between theprocessor 901 and thememory 902 is not limited in the embodiment of the present invention, and theprocessor 901 and thememory 902 are connected through a bus in fig. 9 as an example. The bus may be divided into an address bus, a data bus, a control bus, etc.

In the embodiment of the present invention, thememory 902 stores instructions executable by the at least oneprocessor 901, and the at least oneprocessor 901 may execute the steps included in the foregoing data processing method by executing the instructions stored in thememory 902.

Theprocessor 901 is a control center of the terminal device, and may connect various parts of the terminal device by using various interfaces and lines, and implement data processing by executing or executing instructions stored in thememory 902 and calling data stored in thememory 902. Optionally, theprocessor 901 may include one or more processing units, and theprocessor 901 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application program, and the like, and the modem processor mainly processes an issued instruction. It will be appreciated that the modem processor described above may not be integrated into theprocessor 901. In some embodiments, theprocessor 901 and thememory 902 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.

Theprocessor 901 may be a general-purpose processor, such as a Central Processing Unit (CPU), a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, and may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present invention. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the data processing embodiments may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.

Memory 902, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. TheMemory 902 may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charge Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and so on. Thememory 902 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. Thememory 902 of embodiments of the present invention may also be circuitry or any other device capable of performing a storage function to store program instructions and/or data.

Based on the same technical concept, an embodiment of the present invention provides a backend device, as shown in fig. 10, including at least oneprocessor 1001 and amemory 1002 connected to the at least one processor, where a specific connection medium between theprocessor 1001 and thememory 1002 is not limited in the embodiment of the present invention, and theprocessor 1001 and thememory 1002 in fig. 10 are connected through a bus as an example. The bus may be divided into an address bus, a data bus, a control bus, etc.

In the embodiment of the present invention, thememory 1002 stores instructions executable by the at least oneprocessor 1001, and the at least oneprocessor 1001 may execute the steps included in the foregoing data processing method by executing the instructions stored in thememory 1002.

Theprocessor 1001 is a control center of the backend device, and may connect various parts of the backend device by using various interfaces and lines, and implement data processing by executing or executing instructions stored in thememory 1002 and calling data stored in thememory 1002. Optionally, theprocessor 1001 may include one or more processing units, and theprocessor 1001 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, an application program, and the like, and the modem processor mainly parses a received instruction and parses a received result. It will be appreciated that the modem processor described above may not be integrated into theprocessor 1001. In some embodiments, theprocessor 1001 and thememory 1002 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.

Theprocessor 1001 may be a general-purpose processor, such as a Central Processing Unit (CPU), a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, configured to implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present invention. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the data processing embodiments may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.

Memory 1002, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. TheMemory 1002 may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charge Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and so on. Thememory 1002 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. Thememory 1002 of embodiments of the present invention may also be circuitry or any other device capable of performing a storage function to store program instructions and/or data.

It should be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (13)

Translated fromChinese

1.一种数据处理方法，其特征在于，应用于联邦服务器，所述方法包括：1. A data processing method, characterized in that, applied to a federation server, the method comprising:根据各个参与方设备在公共切割点处的待训练数据的特征维度，确定所述公共切割点处的公共切割特征维度；所述公共切割点是在对所述各个参与方设备的待训练数据进行切割时联合所述各个参与方设备确定出的统一的切割节点；The common cutting feature dimension at the common cutting point is determined according to the feature dimension of the data to be trained at the common cutting point of each participant's device; When cutting, a unified cutting node determined in conjunction with the equipment of each participant;将所述公共切割点处的公共切割特征维度下发给所述各个参与方设备，根据所述各个参与方设备在公共特征维度下的切割特征值，确定公共切割特征值；delivering the common cutting feature dimension at the common cutting point to the respective participant devices, and determining the common cutting feature value according to the cutting feature values of the respective participant devices under the common feature dimension;根据所述公共切割点处的公共切割特征，构建得到检测模型；所述公共切割特征为对异常数据与正常数据进行区分的特征；所述公共切割特征包括公共切割特征维度和公共切割特征值；According to the common cutting feature at the common cutting point, a detection model is constructed; the common cutting feature is a feature that distinguishes abnormal data from normal data; the common cutting feature includes a common cutting feature dimension and a common cutting feature value;将所述检测模型下发给所述各个参与方设备；所述各个参与方设备还用于使用所述检测模型对待检测数据进行检测，以确定所述待检测数据是否为异常数据。The detection model is delivered to the respective participant devices; the respective participant devices are further configured to use the detection model to detect the data to be detected, so as to determine whether the data to be detected is abnormal data.2.根据权利要求1所述的方法，其特征在于，所述参与方设备还用于基于所述公共切割点处的公共切割特征维度确定所述参与方设备在所述公共切割特征维度下的切割特征值；2 . The method according to claim 1 , wherein the participant device is further configured to determine the dimension of the participant device under the common cutting feature dimension based on the common cutting feature dimension at the common cutting point. 3 . cutting eigenvalues;所述根据所述各个参与方设备在公共特征维度下的切割特征值，确定公共切割特征值之前，还包括：Before determining the common cutting feature value according to the cutting feature value of each participant device under the common feature dimension, the method further includes:接收所述各个参与方设备上报的所述各个参与方设备在所述公共切割特征维度下的切割特征值。Receive the cutting feature values of the respective participant devices under the common cutting feature dimension reported by the respective participant devices.3.根据权利要求2所述的方法，其特征在于，所述根据所述各个参与方设备在所述公共切割点处的待训练数据的特征维度，确定所述公共切割点处的公共切割特征维度，包括：3 . The method according to claim 2 , wherein the common cutting feature at the common cutting point is determined according to the feature dimension of the data to be trained at the common cutting point of each participant device. 3 . dimensions, including:根据所述各个参与方设备在所述公共切割点处的待训练数据的特征维度，确定所述各个参与方设备在所述公共切割点处的待训练数据的公共特征维度；从所述公共特征维度中选取所述公共切割特征维度。According to the feature dimension of the data to be trained at the common cut point of the respective participant devices, determine the common feature dimension of the data to be trained at the common cut point of the respective participant devices; from the common feature The common cutting feature dimension is selected from the dimensions.4.根据权利要求1至3中任一项所述的方法，其特征在于，所述根据所述公共切割点处的公共切割特征，构建得到检测模型，包括：4. The method according to any one of claims 1 to 3, characterized in that, constructing a detection model according to the common cutting feature at the common cutting point, comprising:关联任一公共切割点与所述公共切割点处的公共切割特征；associating any common cutting point with the common cutting feature at the common cutting point;根据所述各个参与方设备在所述各个公共切割点处的待训练数据的包含关系，连接所述各个公共切割点，得到二叉树模型，将所述二叉树模型作为所述检测模型。According to the inclusion relationship of the data to be trained at the respective common cut points of the respective participant devices, the respective common cut points are connected to obtain a binary tree model, and the binary tree model is used as the detection model.5.根据权利要求1至3中任一项所述的方法，其特征在于，还包括：5. The method according to any one of claims 1 to 3, further comprising:联合各个参与方设备在任一次训练中的待训练数据，确定所述各个参与方设备在所述任一次训练对应的公共切割点处的公共切割特征，根据所述公共切割点处的公共切割特征，构建得到所述任一次训练对应的检测模型；Combine the data to be trained in any training of each participant's equipment to determine the common cutting feature of each participant's equipment at the common cutting point corresponding to the any training, and according to the common cutting feature at the common cutting point, constructing and obtaining the detection model corresponding to any one of the trainings;所述将所述检测模型下发给所述各个参与方设备，包括：The delivering the detection model to the respective participant devices includes:将各次训练对应的检测模型下发给所述各个参与方设备，以使所述各个参与方设备使用所述各次训练对应的检测模型对所述待检测数据进行检测，以确定所述待检测数据是否为异常数据。Sending the detection model corresponding to each training session to each participant device, so that each participant device uses the detection model corresponding to each training session to detect the data to be detected, so as to determine the to-be-detected data. Check whether the data is abnormal data.6.一种数据处理方法，其特征在于，应用于参与方设备，所述方法包括：6. A data processing method, characterized in that, applied to a participant device, the method comprising:接收联邦服务器下发的公共切割点处的公共切割特征维度；所述公共切割点是在对各个参与方设备的待训练数据进行切割时联合所述各个参与方设备确定出的统一的切割节点；Receive the common cutting feature dimension at the common cutting point issued by the federation server; the common cutting point is a unified cutting node determined in conjunction with each participant device when cutting the data to be trained on each participant device;基于所述公共切割点处的公共切割特征维度确定所述参与方设备在所述公共切割特征维度下的切割特征值；将所述参与方设备在所述公共切割特征维度下的切割特征值上报给所述联邦服务器；Determine the cutting feature value of the participant device under the common cutting feature dimension based on the common cutting feature dimension at the common cutting point; report the cutting feature value of the participant device under the common cutting feature dimension to the federated server;接收联邦服务器发送的检测模型；所述检测模型为所述联邦服务器联合各个参与方设备的待训练数据，确定所述各个参与方设备在公共切割点处的公共切割特征，根据所述公共切割点处的公共切割特征构建得到的；所述公共切割特征用于区分异常数据与正常数据；所述公共切割特征包括公共切割特征维度和公共切割特征值；Receive the detection model sent by the federation server; the detection model is that the federated server combines the data to be trained of each participant's device to determine the common cutting feature of each participant's device at the common cutting point, according to the common cutting point The common cutting feature at the place is constructed and obtained; the common cutting feature is used to distinguish abnormal data from normal data; the common cutting feature includes a common cutting feature dimension and a common cutting feature value;使用所述检测模型对待检测数据进行检测，以确定所述待检测数据是否为异常数据。The data to be detected is detected by using the detection model to determine whether the data to be detected is abnormal data.7.根据权利要求6所述的方法，其特征在于，所述公共切割点处的公共切割特征维度为所述联邦服务器根据所述各个参与方设备在所述公共切割点处的待训练数据的特征维度确定的；7 . The method according to claim 6 , wherein the dimension of the common cutting feature at the common cutting point is the difference between the federated server based on the data to be trained at the common cutting point of each participant device. 8 . The feature dimension is determined;所述联邦服务器还用于根据所述各个参与方设备在公共特征维度下的切割特征值，确定所述公共切割特征值。The federation server is further configured to determine the common cutting feature value according to the cutting feature value of each participant device under the common feature dimension.8.根据权利要求6或7任一项所述的方法，其特征在于，所述使用所述检测模型对待检测数据进行检测，以确定所述待检测数据是否为异常数据，包括：8. The method according to any one of claims 6 or 7, wherein the detecting data to be detected using the detection model to determine whether the data to be detected is abnormal data comprises:使用各个公共切割点处的公共切割特征对所述待检测数据进行切割，确定待检测数据最终被切到的公共切割点；Using the common cutting feature at each common cutting point to cut the data to be detected, and determine the common cutting point to which the data to be detected is finally cut;若所述最终被切到的公共切割点对应的权重大于第一预设阈值，则确定所述待检测数据为异常数据，否则确定所述待检测数据为正常数据。If the weight corresponding to the finally cut common cut point is greater than the first preset threshold, it is determined that the data to be detected is abnormal data; otherwise, the data to be detected is determined to be normal data.9.根据权利要求6或7任一项所述的方法，其特征在于，所述接收联邦服务器发送的检测模型，包括：9. The method according to any one of claims 6 or 7, wherein the receiving the detection model sent by the federation server comprises:接收所述联邦服务器发送的各次训练对应的检测模型；Receive the detection model corresponding to each training sent by the federated server;所述使用所述检测模型对待检测数据进行检测，以确定所述待检测数据是否为异常数据，包括：The using the detection model to detect the data to be detected to determine whether the data to be detected is abnormal data includes:使用任一次训练对应的检测模型中的各个公共切割点处的公共切割特征对待检测数据进行切割，确定待检测数据在所述任一次训练对应的检测模型中最终被切到的公共切割点；Use the common cutting feature at each public cutting point in the detection model corresponding to any training to cut the data to be detected, and determine the public cutting point at which the data to be detected is finally cut in the detection model corresponding to any training;计算所述待检测数据在各次训练对应的检测模型中最终被切到的公共切割点对应的平均权重，若所述平均权重大于第二预设阈值，则确定所述待检测数据为异常数据，否则确定所述待检测数据为正常数据。Calculate the average weight corresponding to the common cutting point that the data to be detected is finally cut in the detection model corresponding to each training, if the average weight is greater than the second preset threshold, then determine that the data to be detected is abnormal data , otherwise it is determined that the data to be detected is normal data.10.一种数据处理装置，其特征在于，所述装置包括：10. A data processing device, wherein the device comprises:确定模块，用于根据各个参与方设备在公共切割点处的待训练数据的特征维度，确定所述公共切割点处的公共切割特征维度；所述公共切割点是在对所述各个参与方设备的待训练数据进行切割时联合所述各个参与方设备确定出的统一的切割节点；A determination module, configured to determine the common cutting feature dimension at the common cutting point according to the characteristic dimension of the data to be trained at the common cutting point of each participant device; When the data to be trained is cut, the unified cutting node determined in conjunction with the equipment of each participant;以及用于将所述公共切割点处的公共切割特征维度下发给所述各个参与方设备，根据所述各个参与方设备在公共特征维度下的切割特征值，确定公共切割特征值；and for issuing the common cutting feature dimension at the common cutting point to the respective participant devices, and determining the common cutting feature value according to the cutting feature values of the respective participant devices under the common feature dimension;构建模块，用于根据所述公共切割点处的公共切割特征，构建得到检测模型；所述公共切割特征为对异常数据与正常数据进行区分的特征；a building module for constructing a detection model according to the common cutting feature at the common cutting point; the common cutting feature is a feature for distinguishing abnormal data from normal data;收发模块，用于将所述检测模型下发给所述各个参与方设备；所述各个参与方设备还用于使用所述检测模型对待检测数据进行检测，以确定所述待检测数据是否为异常数据。a transceiver module, configured to issue the detection model to the respective participant devices; the respective participant devices are further configured to use the detection model to detect the data to be detected, so as to determine whether the data to be detected is abnormal data.11.一种数据处理装置，其特征在于，所述装置包括：11. A data processing device, characterized in that the device comprises:收发模块，用于接收联邦服务器下发的公共切割点处的公共切割特征维度；所述公共切割点是在对各个参与方设备的待训练数据进行切割时联合所述各个参与方设备确定出的统一的切割节点；The transceiver module is used to receive the common cutting feature dimension at the common cutting point issued by the federation server; the common cutting point is determined in conjunction with each participant device when cutting the to-be-trained data of each participant device Unified cutting node;以及用于基于所述公共切割点处的公共切割特征维度确定所述参与方设备在所述公共切割特征维度下的切割特征值；将所述参与方设备在所述公共切割特征维度下的切割特征值上报给所述联邦服务器；And for determining the cutting feature value of the participant device under the common cutting feature dimension based on the common cutting feature dimension at the common cutting point; cutting the participant device under the common cutting feature dimension reporting the characteristic value to the federation server;还用于接收所述联邦服务器发送的检测模型；所述检测模型为所述联邦服务器联合各个参与方设备的待训练数据，确定所述各个参与方设备在公共切割点处的公共切割特征，根据所述公共切割点处的公共切割特征构建得到的；所述公共切割特征用于区分异常数据与正常数据，所述公共切割特征包括所述公共切割特征维度和所述公共切割特征值；It is also used to receive the detection model sent by the federation server; the detection model is that the federated server combines the data to be trained of each participant device to determine the common cutting feature of each participant device at the common cutting point, according to The common cutting feature at the common cutting point is constructed and obtained; the common cutting feature is used to distinguish abnormal data from normal data, and the common cutting feature includes the common cutting feature dimension and the common cutting feature value;检测模块，用于使用所述检测模型对待检测数据进行检测，以确定所述待检测数据是否为异常数据。The detection module is configured to use the detection model to detect the data to be detected, so as to determine whether the data to be detected is abnormal data.12.一种计算设备，其特征在于，包括至少一个处理单元以及至少一个存储单元，其中，所述存储单元存储有计算机程序，当所述程序被所述处理单元执行时，使得所述处理单元执行权利要求1~9任一权利要求所述的方法。12. A computing device, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when the program is executed by the processing unit, causes the processing unit to Execute the method described in any one of claims 1 to 9.13.一种计算机可读存储介质，其特征在于，其存储有可由计算设备执行的计算机程序，当所述程序在所述计算设备上运行时，使得所述计算设备执行权利要求1~9任一权利要求所述的方法。13. A computer-readable storage medium, characterized in that it stores a computer program executable by a computing device, and when the program runs on the computing device, the computing device is made to perform any one of claims 1 to 9. A method as claimed in claim.

Images