CN113886388A - Scene-based identification unifying method, device, equipment and storage medium - Google Patents
Scene-based identification unifying method, device, equipment and storage mediumDownload PDFInfo
- Publication number
- CN113886388A CN113886388ACN202111149627.XACN202111149627ACN113886388ACN 113886388 ACN113886388 ACN 113886388ACN 202111149627 ACN202111149627 ACN 202111149627ACN 113886388 ACN113886388 ACN 113886388A
- Authority
- CN
- China
- Prior art keywords
- field
- data
- scene
- basic
- code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2291—User-Defined Types; Storage management thereof
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a scene-based identification unifying method, a scene-based identification unifying device, a scene-based identification unifying equipment and a scene-based identification unifying storage medium.
Background
The rapid development of artificial intelligence techniques is not open to large-scale computing resources and data resources. The mass computing capability is an effective guarantee for the rapid training of the deep learning model, and the standardized data set is an important basis for carrying out large-scale training and improving the accuracy of the artificial intelligence algorithm. However, these resources are typically distributed across different systems, and users need to access these system data through different data identifications.
However, different data identifiers can be associated with the same data, and a data identifier acquisition method has no uniform standard, so that when data is acquired according to the data identifiers, the data is too dispersed, stability and uniqueness of the data cannot be guaranteed, the data volume is too large, and the storage space is consumed.
Disclosure of Invention
The invention mainly aims to solve the technical problems of overlarge data volume and storage space consumption caused by non-uniform data identification in the prior art.
The first aspect of the present invention provides a method for unifying identifiers based on a scene, where the method for unifying identifiers based on a scene includes: acquiring basic data of each service system and associated data corresponding to each basic data to form a data source in a gathering manner; dividing fields of each basic data in the data source to generate basic fields, and constructing a mapping relation between the basic fields and a preset standard field; coding each basic field according to the mapping relation and a preset field cleaning strategy to obtain field codes; identifying scenes and scene codes in the field codes, and extracting associated data corresponding to the scene codes according to the scene codes; and performing data fusion on the scene codes and the associated data to generate a unique identification.
Optionally, in a first implementation manner of the first aspect of the present invention, the performing field division on each piece of basic data in the data source to generate a basic field, and constructing a mapping relationship between the basic field and a preset standard field includes: carrying out field division on basic data to obtain a basic field, wherein the basic field has a field name and comprises field content; segmenting the field content of the basic field to obtain a plurality of segmentation words; calculating the similarity between the segmentation words and a preset standard field; and constructing a mapping relation between the basic field and the standard field according to the similarity.
Optionally, in a second implementation manner of the first aspect of the present invention, the calculating a similarity between the segmentation word and the preset standard field includes: respectively representing each segmentation word as a word vector; calculating the average value of the word vectors to obtain the central vector of the basic field; and respectively calculating the similarity between the central vector of the basic field and the central vectors of a plurality of preset standard fields.
Optionally, in a third implementation manner of the first aspect of the present invention, the encoding each basic field according to the mapping relationship and a preset field cleaning policy to obtain a field code includes: based on a preset field cleaning strategy, searching a standard code corresponding to each standard field and a special code corresponding to a special field from a preset field code table, wherein the special field is other fields which are not standard fields; according to the mapping relation between the basic fields and the standard fields, using each standard code as the code of each basic field; extracting the rest fields which do not have the mapping relation with the standard fields in the basic fields; encoding the special word as an encoding of the remaining field; and generating a field code corresponding to the basic field according to the codes of the basic fields and the codes of the residual fields.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the identifying a scene and a scene code in the field code, and extracting associated data corresponding to the scene code according to the scene code includes: extracting a company client number scene in the basic data, identifying a company client number code corresponding to the company client number scene from the field codes, and extracting first associated data corresponding to the company client number from the basic data; or, extracting a company mobile phone number scene in the basic data, identifying a company mobile phone number code corresponding to the company mobile phone number scene from the field codes, and extracting second associated data corresponding to the company mobile phone number from the basic data.
Optionally, in a fifth implementation manner of the first aspect of the present invention, when the scene code is the company client number code, the performing data fusion on the scene code and the associated data to generate a unique identification includes: identifying each field code in the first associated data, and judging whether each field code in the first associated data has a control message protocol identification number or not; if so, taking the control message protocol identification number as a unique identification; and if not, creating the unique identification.
Optionally, in a sixth implementation manner of the first aspect of the present invention, when the scene code is the company mobile phone number code, the performing data fusion on the scene code and the associated data, and generating the unique identification includes: identifying each field code in the second associated data, and judging whether each field code in the second associated data has a control message protocol identification number or not; if so, taking the control message protocol identification number as a unique identification; and if not, creating the unique identification.
A second aspect of the present invention provides a scene-based identity unification apparatus, where the scene-based identity unification apparatus includes: the acquisition module is used for acquiring basic data of each service system and relevant data corresponding to each basic data to form a data source in a gathering manner; the dividing module is used for carrying out field division on each basic data in the data source, generating a basic field and constructing a mapping relation between the basic field and a preset standard field; the encoding module is used for encoding each basic field according to the mapping relation and a preset field cleaning strategy to obtain a field code; the identification module is used for identifying scenes and scene codes in the field codes and extracting associated data corresponding to the scene codes according to the scene codes; and the generating module is used for performing data fusion on the scene codes and the associated data to generate a unique identification.
Optionally, in a first implementation manner of the second aspect of the present invention, the dividing module includes: the dividing unit is used for carrying out field division on basic data to obtain a basic field, wherein the basic field has a field name and comprises field content; the word segmentation unit is used for segmenting the field content of the basic field to obtain a plurality of segmentation words; the calculating unit is used for calculating the similarity between the segmentation words and a preset standard field; and the construction unit is used for constructing the mapping relation between the basic field and the standard field according to the similarity.
Optionally, in a second implementation manner of the second aspect of the present invention, the calculating unit includes: the characterization subunit is used for characterizing each segmentation word into a word vector; the first calculating subunit is used for calculating the average value of the word vectors to obtain the central vector of the basic field; and the second calculating subunit is used for respectively calculating the similarity between the central vector of the basic field and the central vectors of the plurality of preset standard fields.
Optionally, in a third implementation manner of the second aspect of the present invention, the encoding module includes: the searching unit is used for searching a standard code corresponding to each standard field and a special code corresponding to a special field from a preset field code table based on a preset field cleaning strategy, wherein the special field is other fields which are not the standard field; a first encoding unit, configured to use each standard code as a code of each basic field according to a mapping relationship between the basic field and the standard field; the extraction unit is used for extracting the residual fields which do not have the mapping relation with the standard fields in the basic fields; a second encoding unit configured to encode the special word as an encoding of the remaining field; and the generating unit is used for generating the field codes corresponding to the basic fields according to the codes of the basic fields and the codes of the residual fields.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the identification module is specifically configured to: extracting a company client number scene in the basic data, identifying a company client number code corresponding to the company client number scene from the field codes, and extracting first associated data corresponding to the company client number from the basic data; or, extracting a company mobile phone number scene in the basic data, identifying a company mobile phone number code corresponding to the company mobile phone number scene from the field codes, and extracting second associated data corresponding to the company mobile phone number from the basic data.
Optionally, in a fifth implementation manner of the second aspect of the present invention, when the scene code is the company client number code, the generating module is specifically configured to: identifying each field code in the first associated data, and judging whether each field code in the first associated data has a control message protocol identification number or not; if each field code in the first associated data has a control message protocol identification number, taking the control message protocol identification number as a unique identification; and if the field codes in the first associated data do not have control message protocol identification numbers, creating unique identification marks.
Optionally, in a sixth implementation manner of the second aspect of the present invention, when the scene code is the company mobile phone number code, the generating module is specifically configured to: identifying each field code in the second associated data, and judging whether each field code in the second associated data has a control message protocol identification number or not; if each field code in the second associated data has a control message protocol identification number, taking the control message protocol identification number as a unique identification; and if the field codes in the second associated data do not have control message protocol identification numbers, creating unique identification marks.
A third aspect of the present invention provides a device for unifying identifiers based on a scene, where the device for unifying identifiers based on a scene includes: a memory having a computer program stored therein and at least one processor, the memory and the at least one processor interconnected by a line; the at least one processor invokes the computer program in the memory to cause the scene-based identity unification device to perform the steps of the scene-based identity unification method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when run on a computer, causes the computer to perform the steps of the above-mentioned scene-based identity unification method.
In the technical scheme provided by the invention, the basic data of each service system and the associated data corresponding to the basic data are obtained, the basic data are subjected to field division to generate a basic field, the basic field is encoded to obtain a field code, the scene code in the field code is extracted, the associated data associated with the scene code is obtained to perform data fusion, and the unique identification is generated. The proposal can automatically generate the unique identification mark of the data, thereby ensuring the stability of the data and the uniqueness of the data identification; the data islands of all the service systems can be communicated through the unique identification marks, and data communication is achieved.
Drawings
Fig. 1 is a schematic diagram of a first embodiment of a scene-based identity unification method in the embodiment of the present invention;
FIG. 2 is a diagram of a second embodiment of a method for unifying scene-based identifications in an embodiment of the present invention;
FIG. 3 is a diagram of a third embodiment of a method for unifying identifiers based on scenarios in an embodiment of the present invention;
FIG. 4 is a diagram of a fourth embodiment of a method for unifying scene-based identifications in an embodiment of the present invention;
FIG. 5 is a diagram of an embodiment of a device for identifying a uniform scene based on a scene according to an embodiment of the present invention;
FIG. 6 is a diagram of another embodiment of a device for identifying a uniform scene based on a scene according to an embodiment of the present invention;
fig. 7 is a schematic diagram of an embodiment of a device for identifying a uniform scene according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a device, equipment and a storage medium for unifying identifiers based on scenes. The proposal can automatically generate the unique identification mark of the data, thereby ensuring the stability of the data and the uniqueness of the data identification; the data islands of all the service systems can be communicated through the unique identification marks, and data communication is achieved.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of understanding, the following describes specific contents of an embodiment of the present invention, and with reference to fig. 1, a first embodiment of a method for unifying scene-based identifiers in an embodiment of the present invention includes:
101, acquiring basic data of each service system and associated data corresponding to each basic data to form a data source;
the server acquires basic data generated by each service system in each service scene, acquires associated data corresponding to the basic data of each service system, and converges the basic data and the associated data to form a data source. In the embodiment of the invention, the basic data comprises service data corresponding to service entities such as users and equipment.
Specifically, user IDs, user data and service data such as a company A client number, a group client number, an APP user number, a company B user ID, a company C mobile phone number and the like are collected to form a data source, and the data source is landed in a database for data analysis of a server.
In addition, the embodiment of the invention can acquire and process the basic data and the associated data generated by the service system based on the artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
102, performing field division on each basic data in the data source to generate a basic field, and constructing a mapping relation between the basic field and a preset standard field;
the method comprises the steps of carrying out field division on each basic data in a data source according to a preset field division rule to generate a basic field, mapping the basic field of the data source to a unified field according to a preset field mapping strategy, wherein in different service systems, the field names are different, some service systems are called names, some service systems are called client _ names, and the field mapping is to map the field names to the unified field names, wherein the unified field names are preset standard fields, and the field mapping is to construct a mapping relation between the basic field and the preset standard fields.
103, coding each basic field according to the mapping relation and a preset field cleaning strategy to obtain a field code;
and according to the constructed mapping relation between the basic fields and the standard fields, the server carries out standardized coding on each basic field according to a preset field cleaning strategy to obtain corresponding field codes. The preset field cleaning strategy is to standardize the code of each field value. For example, in the field of gender, a male value is set as M, a female value is set as F, and other values are NULL, point, NULL, and the like, and then cleaning is performed uniformly according to a field cleaning strategy, for example, NULL, and values of abnormal values are set as "99" and the like.
104, identifying scenes and scene codes in the field codes, and extracting associated data corresponding to the scene codes according to the scene codes;
and 105, performing data fusion on the scene codes and the associated data to generate a unique identification.
The server identifies scenes and scene codes in the field codes according to the relation between each service scene and the corresponding basic data, and extracts associated data corresponding to the scene codes from the data source according to the scene codes, wherein the scenes comprise company client numbers and company mobile phone numbers. The server performs data interchange on the scene codes and the associated data corresponding to the scene codes to generate unique identification identifiers (ONEIDs). Specifically, the server searches the ONEID from the associated data, and if the fields in the associated data are coded and have no ONEID, a new ONEID is created; and if the ONEID exists in each field code in the associated data, using the ONEID as a unique identification mark, and uniformly associating the corresponding data to the ONEID to realize data fusion.
In this embodiment, data merging may be performed according to a unique identification identifier (ONEID), that is, according to the ONEID, reliability ranking is performed in combination with data sources output by each service system, so as to generate a client basic information table with the ONEID as a uniform identifier, which is applied to subsequent client operation work. In this embodiment, ONEID refers to unified data extraction, and is a set of ideas and methods for solving the data islanding problem. Data islanding is a commonly encountered problem after enterprises have developed to a certain stage. Each department, business and product defines and stores data thereof, so that the data are difficult to be correlated and become the existence of an island generally. The ONEID method breaks a data isolated island through unified entity identification and connection, and realizes data integration. In brief, business entities such as users and devices are mapped to a Unique Identifier (UID) in corresponding business data, and data of each dimension is associated through the UID. The UIDs of the service entities are defined and realized differently by various departments, services and products, so that the data cannot be directly related and become a data island. Based on information such as mobile phone numbers, identity cards, mailboxes and equipment IDs, and algorithms such as business rules, machine learning and graph algorithms, ID-Mapping (user unique identification) is carried out, and various UIDs are mapped to a unified ID (ONEID). Through the uniform ID, data of each data island can be associated, data fusion is realized, and accuracy and comprehensiveness of data application such as service analysis, user portrait and the like are ensured.
In the embodiment of the invention, the basic field is formed by carrying out field division on the basic data, the mapping relation between the basic field and the standard field is constructed to carry out field coding on the basic field, and data fusion is carried out according to the field coding to generate the unique identification. The embodiment of the invention can automatically generate the unique identification mark of the data, thereby ensuring the stability of the data and the uniqueness of the data identification; the data islands of all the service systems can be communicated through the unique identification marks, and data communication is achieved.
Referring to fig. 2, a second embodiment of the method for unifying identifiers based on scenes according to the embodiment of the present invention includes:
201, acquiring basic data of each service system and associated data corresponding to each basic data to form a data source;
the server acquires basic data generated by each service system in each service scene, acquires associated data corresponding to the basic data of each service system, and converges the basic data and the associated data to form a data source. In the embodiment of the invention, the basic data comprises service data corresponding to service entities such as users and equipment.
Specifically, user IDs, user data and service data such as a company A client number, a group client number, an APP user number, a company B user ID, a company C mobile phone number and the like are collected to form a data source, and the data source is landed in a database for data analysis of a server.
202, dividing the basic data into fields to obtain basic fields;
and carrying out field division on the basic data, identifying field names and field contents in the basic fields according to a preset field division rule, dividing the basic data into the fields to form the basic fields, wherein the basic fields have the field names and comprise the field contents. In this embodiment, the basic field may be a text-type field, a numeric-type field, or a combination thereof.
203, segmenting the field content of the basic field to obtain a plurality of segmentation words;
and the server performs word segmentation on the field content of the basic field according to a preset word segmentation algorithm to obtain a plurality of segmented words. Word segmentation algorithms can be various, for example: a word segmentation method based on dictionary and word bank matching; the word segmentation method based on word frequency statistics or the word segmentation method based on knowledge understanding, and the like, may also adopt other methods that can segment words for the field to be verified, and are not limited herein.
204, calculating the similarity between the segmentation words and a preset standard field;
205, constructing a mapping relation between the basic field and the standard field according to the similarity;
the server calculates the similarity between the segmentation words and the preset standard fields, compares the similarity with the preset similarity threshold value, and establishes a mapping relation between the segmentation words with the similarity not less than the similarity threshold value and the standard fields, so that the mapping relation between the segmentation words in the basic fields and the standard fields can be established according to the similarity between the segmentation words and the standard fields.
The server adopts a preset word2vec model or a neural network language model and other language models as a characterization tool, and can also determine a word vector corresponding to each segmentation word by inquiring a mapping relation between the segmentation word and the word vector which is established in advance, so that each segmentation word is characterized as a word vector, and each word vector corresponds to each segmentation word one by one, so that the text characteristics of the basic field can be digitalized, and the similarity between the basic field and the preset standard field can be more accurately compared.
And determining the standard field which is most similar to the basic field by comparing the similarity of the central vector of each segmentation word and the central vector of each standard field. The server calculates the average of all word vectors, which is the center vector of the base field. In this embodiment, the central vector can more comprehensively reflect the characteristics of the basic field, which is beneficial to improving the accuracy of similarity comparison.
And calculating the average value of all the standard fields, taking the average value as the central vector of the standard fields, and respectively calculating the similarity between the central vector of the basic field and the central vector of each preset standard field. In this embodiment, the similarity may be a cosine similarity, an adjusted cosine similarity, or a pearson correlation coefficient, which may be used to determine the similarity between vectors.
206, coding each basic field according to the mapping relation and a preset field cleaning strategy to obtain a field code;
207, identifying scenes and scene codes in the field codes, and extracting associated data corresponding to the scene codes according to the scene codes;
and 208, performing data fusion on the scene codes and the associated data to generate a unique identification.
In the embodiment of the present invention, the steps 206-208 are the same as the steps 103-105 in the first embodiment of the scene-based identity unifying method, and are not described herein again.
In the embodiment of the invention, the field division and the word segmentation processing are carried out on the basic data to obtain the segmentation words, and the similarity between the segmentation words and the standard field is calculated, so that the mapping relation between the basic field and the standard field is constructed according to the similarity, the accuracy of the construction of the mapping relation is improved, and the subsequent encoding and data fusion of the basic data according to the mapping relation are facilitated.
Referring to fig. 3, a third embodiment of the method for unifying identifiers based on scenes in the embodiments of the present invention includes:
301, acquiring basic data of each service system and associated data corresponding to each basic data to form a data source;
302, performing field division on each basic data in the data source to generate a basic field, and constructing a mapping relation between the basic field and a preset standard field;
303, encoding each basic field according to the mapping relation and a preset field cleaning strategy to obtain a field code;
and the server searches the standard codes corresponding to the standard fields and the special codes corresponding to the special fields from a preset field code table by adopting a preset field cleaning strategy. In this embodiment, the field cleaning strategy is to encode and standardize each basic field; the special field is other fields which are not standard fields; each standard field and the code corresponding to each standard field are specified in a preset field code table.
And the server takes each standard code as the code of each basic field according to the mapping relation between the basic field and the standard field. For example, in the field of gender, a male value is M, a female value is F, and each basic field is encoded according to the code of the corresponding standard field to form the code of each basic field.
And extracting the residual fields which do not have the mapping relation with the standard fields in the basic fields, wherein in the embodiment, the basic fields have special fields which do not have the corresponding relation with the standard fields, and the special fields comprise abnormal values and null values, and then extracting the special fields in the basic fields and collecting all the special fields to form the residual fields. And taking the special codes corresponding to the special fields in the field code table as codes of all the remaining fields, for example, taking abnormal values as '99', taking other fields as NULL, taking points as NULL, and the like.
And the server generates a field code corresponding to the basic field according to the codes of all the basic fields and the codes of the rest fields in the basic fields, namely the field code is the code corresponding to the basic field.
304, extracting a company client number scene in the basic data, identifying a company client number code corresponding to the company client number scene from the field codes, and extracting first associated data corresponding to the company client number from the data source;
each data in the basic data corresponds to each scene, wherein the scenes comprise a company client number scene and a company mobile phone number scene. The server extracts a company client number scene in the base data, identifies a company client number code corresponding to the company client number scene from the field codes, and extracts first associated data corresponding to the company client number code from the data source. For example: company a customer number scenario can extract the following associated data from the data source:
company a customer number (non-null) company a customer number;
company a client number (non-null) five items (which may be null);
company a customer number (non-null) group customer number (may be null);
company a customer number (non-null) company a mobile phone number (which may be null).
In this embodiment, the first related data is not limited, and may be set according to actual situations.
305, identifying each field code in the first associated data, and judging whether each field code in the first associated data has a control message protocol identification number;
306, if the field codes in the first associated data have control message protocol identification numbers, taking the control message protocol identification numbers as unique identification marks;
307, if the field codes in the first associated data do not have the control message protocol identification number, creating a unique identification.
When the scene code is a company client number code, performing data interchange on the company client number code and first associated data corresponding to the company client number code. The server identifies and extracts each field code in the first associated data, and judges whether each field code in the first associated data has a control message protocol identification number (ICPID), namely, compares each field code, checks whether each field code in the first associated data is an ICPID, namely, judges whether each field code has a unique identification (ONEID).
And if the field code in the first associated data is ICPID, using the ICPID as a unique identification (ONEID), and updating the association of the data according to the ICPID, namely associating the data corresponding to the company client number code to the ICPID.
If no field code in the first associated data is ICPID, a unique identification mark (ONEID) is created, and the data corresponding to the company client number code is associated to the ONEID.
For example, the ICMPID IS searched from each field code according to the company client number A, the five-item ID1 AND the group client number AP1 respectively, if the ICMPID IS not found, namely (ONEID _ BY _ A company client number A IS NULL) AND (ONEID _ BY _ five-item ID1 IS NULL) AND (ONEID _ BY _ group client number AP1 IS NULL), a client IS newly identified; a new ONEID _01 is required to be created, and corresponding data is hung below the ONEID _ 01; the data thus stored are as follows:
(ONEID — 01A company client number a is valid);
(ONEID — 01 five ID1 valid);
(ONEID _01 clique client number AP1 valid);
(ONEID — 01A company mobile phone number P1 is valid);
(ONEID _01A Mobile phone number P2 is valid).
In the embodiment of the present invention, the steps 301-302 are consistent with the steps 101-102 in the first embodiment of the scene-based identity unifying method, and are not described herein again.
In the embodiment of the invention, the unique identification mark is created for the basic data and the associated data of each business system based on the company client number scene, so that data fusion is realized, a data island formed by each business system is broken, and the data of each business system can be conveniently searched and analyzed subsequently.
Referring to fig. 4, a fourth embodiment of the method for unifying identifiers based on scenes according to the embodiments of the present invention includes:
401, acquiring basic data of each service system and associated data corresponding to each basic data, and converging the basic data and the associated data to form a data source;
402, performing field division on each basic data in the data source to generate a basic field, and constructing a mapping relation between the basic field and a preset standard field;
403, encoding each basic field according to the mapping relationship and a preset field cleaning strategy to obtain a field code;
and the server searches the standard codes corresponding to the standard fields and the special codes corresponding to the special fields from a preset field code table by adopting a preset field cleaning strategy. In this embodiment, the field cleaning strategy is to encode and standardize each basic field; the special field is other fields which are not standard fields; each standard field and the code corresponding to each standard field are specified in a preset field code table.
And the server takes each standard code as the code of each basic field according to the mapping relation between the basic field and the standard field. For example, in the field of gender, a male value is M, a female value is F, and each basic field is encoded according to the code of the corresponding standard field to form the code of each basic field.
And extracting the residual fields which do not have the mapping relation with the standard fields in the basic fields, wherein in the embodiment, the basic fields have special fields which do not have the corresponding relation with the standard fields, and the special fields comprise abnormal values and null values, and then extracting the special fields in the basic fields and collecting all the special fields to form the residual fields. And taking the special codes corresponding to the special fields in the field code table as codes of all the remaining fields, for example, taking abnormal values as '99', taking other fields as NULL, taking points as NULL, and the like.
And the server generates a field code corresponding to the basic field according to the codes of all the basic fields and the codes of the rest fields in the basic fields, namely the field code is the code corresponding to the basic field.
404, extracting a company mobile phone number scene in the basic data, identifying a company mobile phone number code corresponding to the company mobile phone number scene from the field codes, and extracting second associated data corresponding to the company mobile phone number code from the data source;
the server extracts the company mobile phone number scene in the basic data, identifies the company mobile phone number code corresponding to the company mobile phone number scene from the field code, and extracts second associated data corresponding to the company mobile phone number code from the data source. For example: the company B mobile phone number scene can extract the following associated data from a data source:
company B mobile phone number (non-null) company B mobile phone number;
company B mobile phone number (non-null) five items (which may be null);
company B mobile phone number (non-null) group customer number (which may be null).
In this embodiment, the second related data is not limited, and may be set according to actual situations.
405, identifying each field code in the second associated data, and judging whether each field code in the second associated data has a control message protocol identification number;
406, if the field codes in the second associated data have control message protocol identification numbers, taking the control message protocol identification numbers as unique identification marks;
407, if the field codes in the second associated data do not have the control message protocol identification number, creating a unique identification.
And when the scene code is the company mobile phone number code, performing data interchange on the company mobile phone number code and second associated data corresponding to the company mobile phone number code. The server identifies and extracts each field code in the second associated data, and judges whether each field code in the second associated data has a control message protocol identification number (ICPID), namely, compares each field code, checks whether each field code in the second associated data is an ICPID, namely, judges whether each field code has a unique identification (ONEID).
And if the field code in the second associated data is ICPID, using the ICPID as a unique identification identifier (ONEID), and updating the association of the data according to the ICPID, namely associating the data corresponding to the mobile phone number code of the company to the ICPID.
If no field code in the second associated data is ICPID, a unique identification mark (ONEID) is created, and the data corresponding to the mobile phone number code of the company are all associated to the ONEID.
For example, the icpid IS searched from each field code according to five items ID1 AND clique client number AP1, if the icpid found out BY five items IS empty, the icpid found out BY clique client number IS not empty, that IS, (ONEID _ BY _ five items ID1 IS NULL) AND (ONEID _ BY _ clique client number AP1 ISNOT NULL), it indicates that a five item IS newly identified; a new ONEID _02 needs to be created, and corresponding data is hung below the ONEID _ 02; meanwhile, setting the relation between the original ONEID and the group client number AP1 as failure; the data thus stored are as follows:
(ONEID _02 five ID1 valid);
(ONEID _02 clique client number AP1 valid);
(ONEID — 02B company mobile phone number P1 is valid);
(ONEID _ BY _ clique client number AP1 clique client number AP1 invalid).
In the embodiment of the present invention, the steps 401-402 are the same as the steps 101-102 in the first embodiment of the scene-based identity unifying method, and are not described herein again.
In the embodiment of the invention, the unique identification mark is created for the basic data and the associated data of each business system based on the mobile phone number scene of the company, so that data fusion is realized, a data island formed by each business system is broken, and the data of each business system can be conveniently searched and analyzed subsequently.
With reference to fig. 5, an embodiment of the device for unifying identifiers based on scenes in the embodiments of the present invention includes:
an obtainingmodule 501, configured to obtain basic data of each service system and associated data corresponding to each basic data, and aggregate the basic data and the associated data to form a data source;
adividing module 502, configured to perform field division on each piece of basic data in the data source, generate a basic field, and construct a mapping relationship between the basic field and a preset standard field;
theencoding module 503 is configured to encode each basic field according to the mapping relationship and a preset field cleaning policy to obtain a field code;
an identifyingmodule 504, configured to identify a scene and a scene code in the field code, and extract associated data corresponding to the scene code according to the scene code;
and agenerating module 505, configured to perform data fusion on the scene code and the associated data, and generate a unique identification.
In the embodiment of the invention, the field division is carried out on the basic data by the scene-based identification unifying device to form the basic field, the mapping relation between the basic field and the standard field is constructed to carry out the field coding on the basic field, the data fusion is carried out according to the field coding, and the unique identification is generated. The embodiment of the invention can automatically generate the unique identification mark of the data, thereby ensuring the stability of the data and the uniqueness of the data identification; the data islands of all the service systems can be communicated through the unique identification marks, and data communication is achieved.
Referring to fig. 6, another embodiment of the device for unifying identifiers based on scenes in the embodiments of the present invention includes:
an obtainingmodule 501, configured to obtain basic data of each service system and associated data corresponding to each basic data, and aggregate the basic data and the associated data to form a data source;
adividing module 502, configured to perform field division on each piece of basic data in the data source, generate a basic field, and construct a mapping relationship between the basic field and a preset standard field;
theencoding module 503 is configured to encode each basic field according to the mapping relationship and a preset field cleaning policy to obtain a field code;
an identifyingmodule 504, configured to identify a scene and a scene code in the field code, and extract associated data corresponding to the scene code according to the scene code;
and agenerating module 505, configured to perform data fusion on the scene code and the associated data, and generate a unique identification.
Wherein thedividing module 502 comprises:
adividing unit 5021, configured to perform field division on basic data to obtain a basic field, where the basic field has a field name and includes field content;
aword segmentation unit 5022, configured to perform word segmentation on the field content of the basic field to obtain multiple segmented words;
the calculatingunit 5023 is used for calculating the similarity between the segmentation words and the preset standard fields;
aconstructing unit 5024, configured to construct a mapping relationship between the basic field and the standard field according to the similarity.
Wherein the calculatingunit 5023 comprises:
acharacterization subunit 50231, configured to characterize each segmented word as a word vector;
a firstcalculating subunit 50232, configured to calculate an average value of the word vectors to obtain a central vector of the basic field;
a secondcalculating subunit 50233, configured to calculate similarities between the center vector of the basic field and the center vectors of the multiple preset standard fields, respectively.
Wherein theencoding module 503 comprises:
a searching unit 5031, configured to search, based on a preset field cleaning policy, a standard code corresponding to each standard field and a special code corresponding to a special field from a preset field code table, where the special field is another field that is not a standard field;
a first encoding unit 5032, configured to use each standard code as a code of each basic field according to a mapping relationship between the basic field and the standard field;
an extracting unit 5033, configured to extract remaining fields in the basic field that do not have a mapping relationship with the standard field;
a second encoding unit 5034 for encoding the special word as an encoding of the residual field;
a generating unit 5035, configured to generate a field code corresponding to the base field according to the code of each of the base fields and the code of the remaining field.
Theidentification module 504 is specifically configured to:
extracting a company client number scene in the basic data, identifying a company client number code corresponding to the company client number scene from the field codes, and extracting first associated data corresponding to the company client number from the basic data;
or, extracting a company mobile phone number scene in the basic data, identifying a company mobile phone number code corresponding to the company mobile phone number scene from the field codes, and extracting second associated data corresponding to the company mobile phone number from the basic data.
Wherein, when the scene code is the company client number code, thegenerating module 505 is specifically configured to:
identifying each field code in the first associated data, and judging whether each field code in the first associated data has a control message protocol identification number or not;
if each field code in the first associated data has a control message protocol identification number, taking the control message protocol identification number as a unique identification;
and if the field codes in the first associated data do not have control message protocol identification numbers, creating unique identification marks.
When the scene code is the company mobile phone number code, thegenerating module 505 is specifically configured to:
identifying each field code in the second associated data, and judging whether each field code in the second associated data has a control message protocol identification number or not;
if each field code in the second associated data has a control message protocol identification number, taking the control message protocol identification number as a unique identification;
and if the field codes in the second associated data do not have control message protocol identification numbers, creating unique identification marks.
In the embodiment of the invention, the scene-based identification unifying device establishes the unique identification for the basic data and the associated data of each business system based on the company client number scene and the company mobile phone number scene, so that the data integration is realized, the data island formed by each business system is broken, and the data of each business system can be conveniently searched and analyzed subsequently.
Referring to fig. 7, an embodiment of a device for identifying and unifying a scene-based identity in the embodiment of the present invention is described in detail below from the perspective of hardware processing.
Fig. 7 is a schematic structural diagram of a scene-based identification unifying apparatus according to an embodiment of the present invention, where the scene-based identificationunifying apparatus 700 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 710 (e.g., one or more processors) and amemory 720, one or more storage media 730 (e.g., one or more mass storage devices) storing anapplication 733 ordata 732.Memory 720 andstorage medium 730 may be, among other things, transient storage or persistent storage. The program stored on thestorage medium 730 may include one or more modules (not shown), each of which may include a series of instruction operations for the scene-basedidentification unification apparatus 700. Still further, theprocessor 710 may be configured to communicate with thestorage medium 730 to execute a series of instruction operations in thestorage medium 730 on the scene-basedidentification unification device 700.
The scene-basedidentification unification device 700 may also include one ormore power supplies 740, one or more wired or wireless network interfaces 750, one or more input-output interfaces 760, and or one ormore operating systems 731, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and so forth. Those skilled in the art will appreciate that the configuration of the scene-based identity unification device illustrated in fig. 7 does not constitute a limitation of the scene-based identity unification device, and may include more or fewer components than those illustrated, or some components in combination, or a different arrangement of components.
The server referred by the invention can be an independent server, and can also be a cloud server for providing basic cloud computing services such as cloud service, a cloud database, cloud computing, cloud functions, cloud storage, Network service, cloud communication, middleware service, domain name service, security service, Content Delivery Network (CDN), big data and artificial intelligence platform and the like.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the scene-based identity unification method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111149627.XACN113886388A (en) | 2021-09-29 | 2021-09-29 | Scene-based identification unifying method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111149627.XACN113886388A (en) | 2021-09-29 | 2021-09-29 | Scene-based identification unifying method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113886388Atrue CN113886388A (en) | 2022-01-04 |
Family
ID=79008021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111149627.XAPendingCN113886388A (en) | 2021-09-29 | 2021-09-29 | Scene-based identification unifying method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113886388A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114780611A (en)* | 2022-04-14 | 2022-07-22 | 北京百度网讯科技有限公司 | A method, device, device and storage medium for identifying and getting through |
| CN115658972A (en)* | 2022-10-25 | 2023-01-31 | 苏州热工研究院有限公司 | Homomorphic different code data retrieval construction method and system |
| CN116303384A (en)* | 2023-02-17 | 2023-06-23 | 中银金融科技有限公司 | Scene data management method, device, storage medium and equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150205830A1 (en)* | 2014-01-23 | 2015-07-23 | International Business Machines Corporation | Tag management in a tag cloud |
| CN109359171A (en)* | 2018-08-21 | 2019-02-19 | 中国平安人寿保险股份有限公司 | Management-control method, device, computer equipment and the storage medium of label |
| CN110765112A (en)* | 2019-10-29 | 2020-02-07 | 国网电子商务有限公司 | Energy equipment data association method and system based on Internet of things identification technology |
| CN112069774A (en)* | 2020-09-03 | 2020-12-11 | 微医云(杭州)控股有限公司 | A data mapping method, device, electronic terminal and storage medium |
| CN112508707A (en)* | 2021-02-03 | 2021-03-16 | 国网电子商务有限公司 | New energy data aggregation method and transaction system based on identification and block chain |
- 2021
- 2021-09-29CNCN202111149627.XApatent/CN113886388A/enactivePending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150205830A1 (en)* | 2014-01-23 | 2015-07-23 | International Business Machines Corporation | Tag management in a tag cloud |
| CN109359171A (en)* | 2018-08-21 | 2019-02-19 | 中国平安人寿保险股份有限公司 | Management-control method, device, computer equipment and the storage medium of label |
| CN110765112A (en)* | 2019-10-29 | 2020-02-07 | 国网电子商务有限公司 | Energy equipment data association method and system based on Internet of things identification technology |
| CN112069774A (en)* | 2020-09-03 | 2020-12-11 | 微医云(杭州)控股有限公司 | A data mapping method, device, electronic terminal and storage medium |
| CN112508707A (en)* | 2021-02-03 | 2021-03-16 | 国网电子商务有限公司 | New energy data aggregation method and transaction system based on identification and block chain |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114780611A (en)* | 2022-04-14 | 2022-07-22 | 北京百度网讯科技有限公司 | A method, device, device and storage medium for identifying and getting through |
| CN115658972A (en)* | 2022-10-25 | 2023-01-31 | 苏州热工研究院有限公司 | Homomorphic different code data retrieval construction method and system |
| CN116303384A (en)* | 2023-02-17 | 2023-06-23 | 中银金融科技有限公司 | Scene data management method, device, storage medium and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113886388A (en) | Scene-based identification unifying method, device, equipment and storage medium | |
| CN105095195B (en) | Nan-machine interrogation's method and system of knowledge based collection of illustrative plates | |
| CN113836038B (en) | Test data construction method, device, equipment and storage medium | |
| CN112632278A (en) | Labeling method, device, equipment and storage medium based on multi-label classification | |
| CN107832287A (en) | A kind of label identification method and device, storage medium, terminal | |
| CN113221570B (en) | Processing method, device, equipment and storage medium based on online inquiry information | |
| CN112559526A (en) | Data table export method and device, computer equipment and storage medium | |
| CN115098650A (en) | Comment information analysis method based on historical data model and related device | |
| CN116108276A (en) | Artificial intelligence-based information recommendation method, device and related equipment | |
| CN117272123B (en) | A sensitive data processing method, device and storage medium based on large models | |
| CN110347827B (en) | Event Extraction Method for Heterogeneous Text Operation and Maintenance Data | |
| WO2023272862A1 (en) | Risk control recognition method and apparatus based on network behavior data, and electronic device and medium | |
| CN113611427A (en) | User portrait generation method, device, equipment and storage medium | |
| CN117785861A (en) | A multi-source heterogeneous data processing method and system | |
| CN113204643A (en) | Entity alignment method, device, equipment and medium | |
| CN119089237B (en) | Refined data processing method based on artificial intelligence | |
| CN110765276A (en) | Entity alignment method and device in knowledge graph | |
| CN112765993B (en) | Semantic parsing method, system, device and readable storage medium | |
| CN113780806A (en) | Broker matching method, device, equipment and storage medium based on decision tree | |
| CN114004228A (en) | Medical text data standardization processing method and device | |
| CN117390933B (en) | Process data tracing method and system for lubricating oil preparation | |
| CN115481084B (en) | BIM model resource management system | |
| CN116340537A (en) | Character relationship extraction method, device, electronic device and storage medium | |
| CN114996386A (en) | Business role identification method, device, equipment and storage medium | |
| CN114547335A (en) | Service data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |