Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Referring to fig. 1, fig. 1 is a flowchart of a field identifying method provided in the present disclosure, as shown in fig. 1, including the following steps:
step S101, identifying active data and inactive data from the dataset.
The data set may be log data of a certain time period. For example: log data accumulated for one month, half year, etc., and the above data set is updated in time, for example: updates are made on a day level. It should be noted that, the data set is not limited to log data in this disclosure, for example: order data is also possible.
The active data may refer to data in which the data set is active in a preset time range, for example, data carrying identification information in the preset time range, where the data may also be understood as data corresponding to an active identification (for example, an identification occurring in the preset time range); the inactive data may be data that is inactive within a preset time range, and these data may also be understood as data corresponding to an inactive identifier (e.g., an identifier that does not appear within the preset time range). The preset time range may be set according to actual requirements, such as one month or half month.
The data in the data set in the present disclosure is data containing identification information, for example: the data comprising the equipment identification, the data comprising the account identification and the data comprising the equipment identification and the account identification. Wherein the device identification may include: an international mobile equipment identity (International Mobile Equipment Identity, IMEI) of the device, a media access control (Medium Access Control, MAC) identification of the device, an advertising identifier (identifier for advertising, IDFA) of the device, an anonymous device identifier (Open Anonymous Device Identifier, OAID) of the device, a COOKIE of the device, and the like; the account identifier may include: mobile phone number, user identification (USERID), microblog software account, chat software account, shopping software account, etc. In the technical scheme of the disclosure, the related data acquisition, storage, application and the like all conform to the regulations of related laws and regulations and do not violate the popular public order.
Step S102, a first association relation of the active data is established, wherein the first association relation is used for representing: the association relation between the equipment identifier and the account identifier and at least one of the association relation between the account identifier and the account identifier.
The first association relationship for establishing the active data may be an association relationship between a device identifier and an account identifier in the active data, or an association relationship between an account identifier and an account identifier, for example: logging in one or more accounts on the same device, so as to establish a first association relationship between the device identification of the device and the account identifications of the accounts; also for example: some account identifications appear in the same piece of log data, so that a first association relationship is established between the account identifications.
The first association relationship is used for representing: the association relationship between the device identifier and the account identifier, and at least one of the association relationship between the account identifier and the account identifier can be understood as: the first association may represent an association between the device identifier and the account identifier, or the first association may represent an association between the account identifier and the account identifier, or the first association may represent an association between the device identifier and the account identifier, and represent an association between the account identifier and the account identifier.
It should be noted that the active data may include a plurality of first association relationships, where different first association relationships correspond to different device identifiers or correspond to different account identifiers.
Step S103, obtaining a second association relationship of the pre-stored inactive data, where the second association relationship is used to represent: at least one of an association relationship between the device identifier and the account identifier and an association relationship between the account identifier and the account identifier.
The second association relationship of the pre-stored inactive data may be a second association relationship pre-established for the inactive data, for example: the second association relationship established for the inactive data two months or one month ago or half month ago is obtained, or the time is not limited, and the pre-stored second association relationship of the inactive data may be the second association relationship of the inactive data used by the last or last N times of execution of aggregation.
In the method, the pre-stored second association relation is directly used for the inactive data, so that the second association relation of the inactive data is not required to be established in each aggregation process, and the aggregation efficiency can be improved.
It should be noted that the inactive data may include a plurality of second association relationships, where different second association relationships correspond to different device identifiers or correspond to different account identifiers.
Step S104, executing aggregation operation based on the first association relation and the second association relation.
The above-mentioned performing an aggregation operation based on the first association relationship and the second association relationship may be that device identifiers and account identifiers that match the same object are aggregated together to form a corresponding aggregation set. The above object may be set according to actual requirements, for example: in some scenarios the object may be an object of a person, a vehicle, a merchant, a school, or a unit.
In the present disclosure, since active data and inactive data of a data set are identified, an aggregation operation is performed on the inactive data using a pre-stored second association relationship, so that an aggregation efficiency may be improved.
Referring to fig. 2, fig. 2 is a flowchart of another field identifying method provided in the present disclosure, as shown in fig. 2, including the following steps:
step S201, identifying active data and inactive data from the dataset.
Step S202, a first association relation of the active data is established, wherein the first association relation is used for representing: the association relation between the equipment identifier and the account identifier and at least one of the association relation between the account identifier and the account identifier.
In some embodiments, the establishing the first association relationship of the active data may be establishing a connection diagram between the device identifier and the account identifier, or establishing a connection diagram between the account identifier and the account identifier, or directly establishing a mapping relationship between the device identifier and the account identifier, or directly establishing a mapping relationship between the account identifier and the account identifier.
Step S203, acquiring a second association relationship of the pre-stored inactive data, where the second association relationship is used to represent: at least one of an association relationship between the device identifier and the account identifier and an association relationship between the account identifier and the account identifier.
Step S204, executing an aggregation operation based on the first association relationship and the second association relationship.
As an optional implementation manner, in a case that the first association relationship represents an association relationship between a device identifier and an account identifier, and the second association relationship represents an association relationship between a device identifier and an account identifier, the performing, based on the first association relationship and the second association relationship, an aggregation operation includes:
based on the first association relation, aggregating the equipment identification and the account identification matched with the same object in the active data to obtain a first aggregation relation;
Based on the second association relation, aggregating the equipment identifier and the account identifier matched with the same object in the inactive data to obtain a second aggregation relation;
and combining the first polymerization relation and the second polymerization relation.
The aggregating the device identifiers and the account identifiers matched with the same object in the active data based on the first association relationship may be that, according to all or part of the first association relationship in the active data, the device identifiers and the account identifiers matched with the same object are identified and aggregated together to form a first aggregation relationship corresponding to each object, where the first aggregation relationship of each object is used to represent the device identifier and the account identifier corresponding to the object. Similarly, the second polymerization relationship described above can be obtained.
It should be noted that, in the present disclosure, each first aggregate relationship corresponds to one object, and therefore, there may be a plurality of first aggregate relationships and a plurality of second aggregate relationships for the active data and the inactive data.
The merging of the obtained first aggregation relationship and the obtained second aggregation relationship may be that the same object aggregation relationship is included in the first aggregation relationship and the second aggregation relationship, so as to obtain a final aggregation result.
In this embodiment, the aggregation relationships of the active data and the inactive data are obtained, and then the active data and the inactive data are combined, so that the accuracy of aggregation can be improved.
In some embodiments, the disclosure is not limited to aggregation by way of the above-described aggregation relationships that result in active data and inactive data, respectively, such as: in some scenes or embodiments, the final aggregate combination can be obtained directly according to the first association relationship and the second association relationship, that is, the aggregate relationship of the active data and the inactive data is not acquired separately.
Optionally, the data set includes a plurality of pieces of data, and the device identifier and the account identifier that match the same object are determined based on at least one of the following:
a plurality of device identifications included in the same piece of data are matched with the same object;
a plurality of equipment identifiers included in the same piece of data are matched with the same object by account identifiers associated with the equipment identifiers;
a plurality of account identifiers included in the same piece of data are matched with the same object;
the same piece of data comprises a plurality of account identifications, and the equipment identifications associated with the account identifications are matched with the same object.
The same piece of data may be the same piece of log data.
In this embodiment, the matching of the identifiers included in the same piece of data to the same object may be implemented, so that the accuracy of aggregation may be improved. Because in practice it is found that the identity appearing in the same piece of data is often the identity of the same object.
It should be noted that, the determination of the identity matching the same object by the above manner is not limited in this disclosure, for example: in some scenes or embodiments, the identification matching the same object can also be determined through the behavior, the position and other data of the data corresponding to the device identification and the account identification.
Optionally, the aggregating the device identifier and the account identifier matched with the same object in the active data based on the first association relationship to obtain a first aggregation relationship includes:
under the condition that the active data comprises the same account number identifier and a plurality of equipment identifiers are associated, a third association relation of the same account number and the plurality of equipment identifiers is established;
when the active data comprises a plurality of equipment identifiers which are matched, a fourth association relation of the equipment identifiers is established;
and under the condition that at least one of the third association relation and the fourth association relation is established, aggregating the equipment identifier and the account identifier matched with the same object in the active data based on the first association relation and at least one of the third association relation and the fourth association relation to obtain a first aggregation relation.
The third association relationship for establishing the same account to associate a plurality of device identifiers may be that a plurality of device identifiers associated with the same account are associated, for example: and associating a plurality of device identifications associated with the same personal identification information (Personally identifiable information, PII) to obtain the third association relationship, wherein the PII can be a preselected account identification.
The establishing the fourth association relationship of the plurality of device identifiers may be that the device identifiers with the plurality of device behaviors and the matched positions are associated to obtain a fourth association relationship.
The above-mentioned aggregating the device identifiers and the account identifiers matched with the same object in the active data based on at least one of the third association relationship and the fourth association relationship and the first association relationship may be that a plurality of associated device identifiers are determined according to the third association relationship, and then account identifiers associated with the device identifiers are determined according to the corresponding first association relationship, and then the device identifiers and the account identifiers are determined to be matched with the same object, so as to aggregate the device identifiers and the account identifiers.
In this embodiment, because the device identifier and the account identifier that match the same object in the active data are aggregated based on the first association relationship and at least one of the third association relationship and the fourth association relationship, accuracy of the first aggregate association may be improved.
Optionally, the aggregating the device identifier and the account identifier matched with the same object in the inactive data based on the second association relationship to obtain a second aggregation relationship includes:
under the condition that the same account number identifier is associated with a plurality of equipment identifiers, establishing a fifth association relationship of the same account number associated with the plurality of equipment identifiers; when the inactive data comprises a plurality of equipment identifiers which are matched, establishing a sixth association relation of the equipment identifiers; under the condition that at least one of the fifth association relationship and the sixth association relationship is established, aggregating the equipment identifier and the account identifier matched with the same object in the inactive data based on the second association relationship and at least one of the fifth association relationship and the sixth association relationship to obtain a second aggregation relationship;
or alternatively
Under the condition that at least one of a seventh association and an eighth association of the inactive data is stored in advance, aggregating equipment identifiers and account identifiers matched with the same object in the inactive data based on the at least one of the seventh association and the eighth association and the second association to obtain a second aggregation relationship;
The seventh association relationship is an association relationship of associating a plurality of device identifiers with the same account in the inactive data, and the eighth association relationship is an association relationship of matching a plurality of device identifiers in the inactive data.
The aggregating the device identifier and the account identifier matched with the same object in the inactive data based on at least one of the fifth association relationship and the sixth association relationship and the second association relationship to obtain the second association relationship may refer to corresponding descriptions of the embodiments of the third association relationship and the fourth association relationship, which are not described herein, and may also improve accuracy of the second association relationship.
In addition, in this embodiment, since the seventh association and the eighth association of the inactive data stored in advance can be used, the aggregation efficiency can be further improved.
As an alternative embodiment, as shown in fig. 2, the method further includes:
step S205, determining a target object identifier aiming at an aggregation set, wherein the aggregation set is used for representing a device identifier and an account identifier matched with the same object, and the target object identifier is the account identifier or the device identifier meeting preset conditions in the aggregation set.
The target object identifier may be an identifier for representing an object, so that each aggregation set represents a device identifier and an account identifier that match the same object, and thus the aggregation set may be represented by the target object identifier.
The preset conditions may be preset according to actual requirements, for example: in some scenes or embodiments, the account number identification or the device identification with the largest total occurrence number in the data set in the aggregation set may be selected, or in other scenes or embodiments, the account number identification or the device identification with the largest occurrence number in the last week and month in the data set in the aggregation set may be selected; or the account identifier or the device identifier meeting the preset condition may be the account identifier or the device identifier with the highest priority, or the account identifier or the device identifier meeting the preset condition may be the most stable account identifier selected first, and the most stable device identifier is selected under the condition that the most stable account identifier is not available, where the stability may be that the number of times of occurrence of the identifier in the data set is stable, for example, the number of times of occurrence is frequent.
In the embodiment, as the target object identification of the aggregation sets is determined, each aggregation set can be rapidly and accurately positioned through the target object identification, so that the aggregation effect is improved.
The improved information aggregation method of the present disclosure is illustrated by fig. 3 and 4 below:
in one embodiment, the following steps may be included as shown in FIG. 3:
step S301, basic data accumulation;
the base data here may be the data set in the above embodiment, and may be the base data obtained by the hour-level accumulation, the antenna accumulation, and the full-order accumulation.
In some scenarios or embodiments, the base data may include 78% inactive data and 22% active data, it being noted that 78% and 22% are only a simple example.
Step S302, lightweight aggregation is adopted for active data.
The lightweight aggregation comprises equipment aggregation related tasks and aggregation to object related tasks, wherein the equipment aggregation related tasks refer to aggregation of equipment layers, namely, the association relation between equipment identifiers and account identifiers is established, and the aggregation to object related tasks refer to aggregation of the equipment identifiers and the account identifiers to corresponding objects.
Step S303, old data aggregation is adopted for the inactive data.
Here, the old data may refer to an association relationship between a pre-stored device identifier and an account identifier (e.g., PII), and the aggregation may be to generate a target object identifier by using the old data, and a specific process may refer to the determination of the target object identifier in the foregoing embodiment, which is not described herein. In addition, the target object identification in this disclosure may also be defined as UDWID or other names.
And step S304, merging the target object identifiers.
The step can combine the target object identification of the active data and the target object identification of the inactive data, and then pool the database, namely storing the combined data into the database.
In another embodiment, the following steps may be included as shown in FIG. 4:
step S401, acquiring an hour-level task.
Step S402, day-level accumulated data.
Step S403, total accumulation and anti-cheating.
Wherein, the total accumulation refers to acquiring active data and inactive data, as shown in fig. 4, including 80% of inactive data and 20% of active data, and the anti-cheating may refer to identifying erroneous data.
Step S404, lightweight aggregation.
The method comprises the following steps:
Connectivity graph computation, device-level anti-cheating, device aggregation, PII association and aggregation to object connectivity graph computation.
The connection graph calculation may be calculating an association relationship between a device identifier and an account identifier of active data, the device-level anti-cheating may be identifying error data at a device aggregation level, the device aggregation may be aggregating account identifiers associated with the same device, the PII association may be associating a plurality of devices associated with the same account identifier, and the aggregation to object connection graph calculation may be aggregating device identifiers and account identifiers matched with the same object.
Step S405, old data aggregation.
This step may be to obtain a pre-stored association of the device identification with the account identification (e.g., PII).
Step S406, aggregating to the object.
The step can aggregate the device identifications and the account identifications of the same object in the device identifications and the account identifications in the active data and the inactive data.
Step S407, pouring.
Referring to fig. 5, fig. 5 is an information aggregation apparatus provided by the present disclosure, as shown in fig. 5, an information aggregation apparatus 500 includes:
an identification module 501 for identifying active data and inactive data from a dataset;
The establishing module 502 is configured to establish a first association relationship of the active data, where the first association relationship is used to represent: the association relation between the equipment identifier and the account identifier and at least one of the association relation between the account identifier and the account identifier;
an obtaining module 503, configured to obtain a second association relationship of the pre-stored inactive data, where the second association relationship is used to represent: at least one of the association relationship between the equipment identifier and the account identifier and the association relationship between the account identifier and the account identifier;
and an aggregation module 504, configured to perform an aggregation operation based on the first association relationship and the second association relationship.
Optionally, in the case where the first association represents an association between a device identifier and an account identifier, and the second association represents an association between a device identifier and an account identifier, as shown in fig. 6, the aggregation module 504 includes:
a first aggregation unit 5041, configured to aggregate, based on the first association relationship, device identifiers and account identifiers that match the same object in the active data, to obtain a first aggregation relationship;
a second aggregation unit 5042, configured to aggregate, based on the second association relationship, device identifiers and account identifiers that match the same object in the inactive data, to obtain a second association relationship;
A merging unit 5043, configured to merge the first aggregation relation and the second aggregation relation.
Optionally, the data set includes a plurality of pieces of data, and the device identifier and the account identifier that match the same object are determined based on at least one of the following:
a plurality of device identifications included in the same piece of data are matched with the same object;
a plurality of equipment identifiers included in the same piece of data are matched with the same object by account identifiers associated with the equipment identifiers;
a plurality of account identifiers included in the same piece of data are matched with the same object;
the same piece of data comprises a plurality of account identifications, and the equipment identifications associated with the account identifications are matched with the same object.
Optionally, the first aggregation unit 5041 is configured to:
under the condition that the active data comprises the same account number identifier and a plurality of equipment identifiers are associated, a third association relation of the same account number and the plurality of equipment identifiers is established;
when the active data comprises a plurality of equipment identifiers which are matched, a fourth association relation of the equipment identifiers is established;
and under the condition that at least one of the third association relation and the fourth association relation is established, aggregating the equipment identifier and the account identifier matched with the same object in the active data based on the first association relation and at least one of the third association relation and the fourth association relation to obtain a first aggregation relation.
Optionally, the second polymerization unit 5042 is configured to include:
under the condition that the same account number identifier is associated with a plurality of equipment identifiers, establishing a fifth association relationship of the same account number associated with the plurality of equipment identifiers; when the inactive data comprises a plurality of equipment identifiers which are matched, establishing a sixth association relation of the equipment identifiers; under the condition that at least one of the fifth association relationship and the sixth association relationship is established, aggregating the equipment identifier and the account identifier matched with the same object in the inactive data based on the second association relationship and at least one of the fifth association relationship and the sixth association relationship to obtain a second aggregation relationship;
or alternatively
Under the condition that at least one of a seventh association and an eighth association of the inactive data is stored in advance, aggregating equipment identifiers and account identifiers matched with the same object in the inactive data based on the at least one of the seventh association and the eighth association and the second association to obtain a second aggregation relationship;
the seventh association relationship is an association relationship of associating a plurality of device identifiers with the same account in the inactive data, and the eighth association relationship is an association relationship of matching a plurality of device identifiers in the inactive data.
Optionally, as shown in fig. 7, the apparatus further includes:
the determining module 505 is configured to determine, for an aggregate set, a target object identifier, where the aggregate set is used to represent a device identifier and an account identifier that match the same object, and the target object identifier is an account identifier or a device identifier that satisfies a preset condition in the aggregate set.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
Fig. 8 illustrates a schematic block diagram of an example electronic device 800 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the respective methods and processes described above, such as an information aggregation method. For example, in some embodiments, the information aggregation method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809. When a computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the information aggregation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the information aggregation method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.