Disclosure of Invention
In order to overcome the problems, the inventor of the invention accumulates over the years of work, researches and analyses are carried out by taking four types of social basic data (personnel hotel accommodation information, personnel civil aviation ticket booking information, personnel departure and entry information and personnel railway transportation information) capable of effectively mining potential relations as departure points through project practice, scattered, disordered and irrelevant person social information is obtained through data reconstruction, centralized processing and statistical operation, the potential relations between persons are obtained, the application effect on special (such as police criminal investigation) fields is higher, and the value of data is improved, so that the invention is completed.
The invention aims to provide the following technical scheme:
(1) an analysis system for mining potential relationships of people based on social basic information is characterized by comprising a presentation system 100, an application system 200 and a data system 300;
the application system 200 includes:
the data source configuration management module 210: the system is used for configuring and receiving data table information of a business database 310 related to social basic information, and comprises the steps of establishing database connection for accessing a remote database, forming a data source table, and configuring information of an access data table, and forming an access data table;
data table access mapping module 220: the attribute field mapping module is used for mapping the data table in the service database 310 and a predefined data model;
the data integration module 230: the system is used for receiving and extracting data of a data table in the business database 310, extracting and transmitting data values of attribute fields to the dynamic track database 330 according to the mapping condition of data table access, and restoring and storing the data through a set data processing program to obtain detailed information of the same person;
the data statistics and analysis module 240 is configured to perform classification statistics according to the data integration result of the data integration module 230, calculate a relationship intimacy degree score between two people according to the relationship intimacy level, and generate a peer report;
the presentation system 100 includes:
the character relationship display module 110 is configured to display the character relationship in the form of a table or a graph by using the inherent attribute information of the character as a search entry and analyzing the result through the search system.
(2) An analysis method for mining potential relationships of people based on social basic information comprises the following steps:
step 1), configuring and receiving data table information of a service database related to social basic information, wherein the data table information comprises establishing database connection for accessing a remote database to form a data source table, and configuring information of an access data table to form an access data table;
step 2), accessing and mapping the data table, and performing attribute field mapping on the data table in the service database and a predefined data model;
step 3), automatically establishing a conversion view for converting the accessed data table into a specified structure in a configuration library according to the information of the data table access mapping, and storing the name of the conversion view into the accessed data table;
step 4), receiving data table data in the extracted service database, extracting and transmitting attribute field data values to a dynamic track database according to the mapping condition of data table access, and reconstructing and storing the data through a set data processing program to obtain detailed information of the same person;
step 5), carrying out classification statistics according to the result of system data integration, calculating the relationship intimacy degree value between two characters according to the relationship intimacy grade, and generating a co-pedestrian report;
and 6) taking the inherent attribute information as a retrieval entry, and displaying the human-object relationship in a form of a table or a graph through the analysis result of the retrieval system.
According to the analysis system and method for mining the potential relationship of the people based on the social basic information, provided by the invention, the following beneficial effects are achieved:
firstly, the invention takes four types of social basic data (personnel hotel accommodation information, personnel civil aviation ticket booking information, personnel entry and exit information and personnel railway transportation information) which can effectively mine potential relations as the starting point, and provides a powerful data base for obtaining the potential relations among the objects.
Second, the invention relates the field in the business database data table with the model field in the predefined data model through the attribute field mapping by the attribute field mapping, so that the field in the business database data table can be recognized by the system, and the field information corresponding to the model field is reserved by taking the predefined data model as a template, the information irrelevant to the model field is eliminated, the data is screened, the data utilization rate is improved, and the complexity of data operation is reduced.
Thirdly, the extracted data is subjected to level identification on the degree of relationship intimacy of the two characters in the same event according to a set peer checking rule, and the degree of relationship intimacy between the two characters is calculated through a relationship intimacy degree value formula on the basis of the occurrence times of various intimacy levels in peer detailed information related to each event type; the potential relation among the people is displayed in a quantitative mode, the information value is higher, and the display effect is more direct.
Fourth, the modules in the invention are matched and connected, potential connections among people are obtained through data reconstruction, centralized processing and statistical operation on scattered, disordered and irrelevant people social information, and the method has a high application effect on special fields (such as police criminal investigation) (such as obtaining partner information by determining that a suspect comes in and out of a guest hall together in a certain time period), and improves the value of the existing data.
Detailed Description
The invention is explained in more detail below with reference to the figures and examples. The features and advantages of the present invention will become more apparent from the description.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
In order to effectively develop data with large-scale effect from the perspective of data storage, management and data analysis to obtain the potential relationship between people, and display the potential relationship in a clear and quantitative manner, the invention provides an analysis system for mining the potential relationship of people based on social basic information, as shown in fig. 1, the analysis system comprises a display system 100, an application system 200 and a data system 300;
the application system 200 includes:
the data source configuration management module 210: which is used to configure the data table information of the service database 310 that is received in association with the social underlying information, including establishing database connections to access remote databases, forming data source tables, and configuring the information of the access data table, forming access data tables.
Specifically, the data source configuration management module 210 includes a data source submodule 211 and a data table submodule 212, where the data source submodule 211 is configured to establish and maintain data source information of an extracted data table; the data table sub-module 212 is configured to establish and maintain relevant information of the extracted data table, i.e., access data table information.
As shown in table 1, the data source table stores data source information, and the data source information includes data source description, database connection information, creation time, and the like. The data source configuration management module 210 completes configuration of each piece of data source information by adding, deleting or modifying.
TABLE 1 data Source Table
| Name of field | Data type | Properties | Description of the invention |
| Unique identification code | Numerical type | Main key | |
| Data source description | Character type | | |
| Database connection information | Character type | | |
| Creation time | Date type | | Default SYS date type |
As shown in table 2, the access data table includes a table name, a table comment, a table unique identification code field, an event type, a view name, an increment field, a maximum value of processed data, a state identifier, a creation time, and the like.
Table 2 access data table
| Name of field | Data type | Properties | Description of the invention |
| Unique identification code | Numerical type | Main key | |
| Data source unique identification code | Numerical type | External key | Primary key of data source table |
| Table name | Character type | | |
| Table annotation | Character type | | |
| Table unique identification code field | Character type | | |
| Event type | Character type | | |
| View name | Character type | | |
| Increment field | Character type | | |
| Processed data maximum | Numerical type | | The maximum primary key value of the processed data is initialized to 0 |
| Status identification | Character type | | 0 is invalid; 1 is effective |
| Creation time | Date type | | Default SYS date type |
Specifically, the data source configuration management module 210, through the data table sub-module 212:
(i) appointing an existing data source, and recording a unique identification code of the data source;
(ii) selecting a data table name under a data source;
(iii) adding a data table annotation to the selected data table;
(iv) assigning a data table unique identification code field;
(v) appointing the event type to which the data sheet belongs (the event type comprises guest accommodation, departure and entry, civil aviation ticket booking and railway transportation);
(vi) specifying a data increment field.
Data table access mapping module 220: for attribute field mapping of the data tables in the service database 310 with the predefined data model, i.e. associating the fields in the data tables in the service database 310 with the model fields in the predefined data model via attribute field mapping, the model fields in the predefined data model can be recognized by the present system.
The data table access mapping module 220 includes a table field mapping configuration sub-module 221 and a table field mapping association sub-module 222:
table field mapping configuration submodule 221: establishing and maintaining a mapping relation between the data table in the service database 310 and the attribute field of the predefined data model, and forming an access table field model mapping table, as shown in table 3;
table field mapping association submodule 222: accessing the mapping information according to the access table field model mapping table, automatically establishing a conversion view (figure 2) for converting the accessed data table into a specified structure in the configuration library, and saving the conversion view name to the access data table (table 2). The fields displayed in the conversion view related to the event type information are the model fields in the predefined data model.
The translated view will serve as the actual data source for the integrated data (the data tables in the service database 310 are the original data sources) and will be named by the uniform attribute field recognized by the system, which facilitates the subsequent data operation.
It should be noted that, for different event types, attribute field mapping information of the access table field model mapping table is inevitably different, that is, for different event types, it is necessary to independently set a corresponding predefined data model, generate a corresponding access table field model mapping table, and thereby obtain conversion views with different view names. For example, when the event type is "guest lodging", the predefined model fields in the data model include guest name, guest certificate number, guest gender, guest birth date, guest country, hotel checked-in, hotel administrative division checked-in, hotel water flow number checked-in, room number checked-in, time checked-in, and time checked-out. The model fields of the event types "inbound and outbound", "civil aviation booking" and "rail transport" under the respective predefined data models are also shown in the corresponding conversion view in fig. 2.
Table 3 access table field model mapping table
| Name of field | Data type | Properties | Description of the invention |
| Unique identification code | Numerical type | Main key | |
| Unique identification code for access data table | Numerical type | External key | |
| Name of field | Character type | | Data tables in the service database 310 |
| Field description | Character type | | |
| Type of field | Character type | | |
| Model field | Character type | | Mapped field names |
| Creation time | Date type | | Default SYS date type |
It is known that even enterprises operating the same business can express different data information of the business, which is reflected in the field names and the field numbers of the data tables.
For the field names, the data table access mapping module 220 obtains model fields that can be recognized by the system through attribute field mapping.
For the number of fields, the data table access mapping module 220 obtains a part of fields with higher value and higher correlation with the event type in the original data table based on the predefined model fields in the data model, and does not map fields with less correlation with the event type, thereby realizing data screening, which provides an effective data basis for subsequent data integration.
In a preferred embodiment, the data table access mapping module 220 further includes a data type checking sub-module 223, which performs data type checking according to the definition condition of the attribute field in the data table in the service database 310, and performs labeling or other processing on the field information that does not meet the definition condition, and is not used in the subsequent integration processing.
The data integration module 230: the data processing system is used for receiving data table data in the extracted service database 310, extracting and transmitting attribute field data values to the dynamic track database 330 according to the mapping condition of data table access, and reproducing and storing the data through a set data processing program to acquire the detailed information of the same person.
The set data processing program is used for verifying the intimacy degree of the corresponding field information of any two persons in the same event type data table according to the set peer rule, and performing level identification on the relationship intimacy degree of the two persons in the same event (such as the entrance to the same hotel). The pedestrian-by-pedestrian rule is shown in table 4, and the intimacy degree classification criterion is shown in table 5, for example.
TABLE 4 Council rules
TABLE 5 intimacy rating Scale
| Identification code | Event type | Rule description of co-walking person | Intimacy rating | Thin type identification code |
| 1 | Accommodation device | Enter the same hotel on the same day and have the same serial number | A | 1 |
| 2 | Accommodation device | Enter the same hotel on the same day and have the same room number | A | 2 |
| 3 | Accommodation device | Enter the same hotel on the same day and have the same group mark | A | 3 |
| 4 | Accommodation device | Check-in and check-out (difference of 10 minutes) | B | 4 |
| 5 | Entry and exit | The same port is accessed on the same day and the group marks are the same | A | 1 |
| 6 | Entry and exit | Enter and exit the same port on the same day with the difference of the entering and exiting time of 10 minutes | B | 2 |
| 7 | Civil aviation | Take the same flight on the same day and have the same booking number | A | 1 |
| 8 | Civil aviation | Take the same flight on the same day and have the same group mark | A | 2 |
| 9 | Railway track | The same train number and the same group identification on the same day | A | 1 |
| 10 | Railway track | The same train number and the same starting station and the same arrival station on the same day | B | 2 |
| 11 | Railway track | The same train number and the same starting station or arriving station on the same day | C | 3 |
| 12 | Railway track | Same train number on same day | D | 4 |
| …… | …… | …… | …… | …… |
Specifically, in the present invention, the data integration module 230 includes:
a data extraction submodule 231 for starting an extraction procedure of the data table in the service database 310; the data extraction submodule 231 can automatically operate through the data extraction time set in the system, and the system automatic operation parameters are shown in a system dictionary in table 6;
the data loading submodule 232 transmits the data value of the mapped service library to the dynamic track library 330 according to the mapping rule of the field model mapping table of the access table, and generates the details of the same person, as shown in table 7;
the data conversion sub-module 233 reproduces and stores data by a set data processing program. The set data processing program is used for verifying the intimacy degree of corresponding field information of any two persons in the same event type data table according to set peer rules and performing level identification on the intimacy degree of the relationship of the two persons in the same event;
a log record submodule 234, configured to record operation conditions generated in each data extraction, loading, and conversion process, and form a log record, as shown in table 8;
the data management submodule 235 is used for displaying the updating condition of the data of the system every day through the log record generated in the data integration;
the operation monitoring sub-module 236 tracks the operation status of the data integration module 230 through the log records generated in the data integration, and alarms in a message box mode when an abnormality occurs.
TABLE 6 System dictionary
| Name of field | Data type | Properties | Description of the invention |
| Unique identification code | Numerical type | Main key | |
| Segment of | Character type | | |
| ID | Character type | | |
| Value of | Character type | | |
| Status identification | Character type | | 0 is invalid; 1 is effective |
TABLE 7 detailed information table of the fellow pedestrians
TABLE 8 Log records
| Name of field | Data type | Properties | Description of the invention |
| Unique identification code | Numerical type | Main key | |
| Time of generation | Date type | | |
| Log categories | Character type | | |
| Name of interface program | Character type | | |
| Log content | Character type | | |
| Status of state | Character type | | Default SYS date type |
In the invention, because the information amount in the data table under the same type is huge, or the data in the data table is updated, or due to other situations, the system can not complete the integration all at once during the data integration.
The system of the invention allows for multiple integrations for the above situations. The data integration module 230 further comprises the following operations:
the data extraction submodule 231 is used for acquiring data information of an effective state in the access data table, wherein important information comprises a view name (namely a converted view name), an increment field and a processed data maximum value;
and the data loading submodule 232 determines a loading range of the access data according to the increment field set by the access data table and the maximum value of the processed data, wherein the starting point is the maximum value of the processed data, the end point is the maximum value of the increment field in the current view, after the loading is finished, the maximum value of the processed data is recorded, and the maximum value of the processed data is stored in the access data table to be used as the starting value of the next data loading.
In a preferred embodiment, when the data conversion sub-module 233 operates, the set data processing program performs affinity verification on the corresponding field information of any two people through an embodiment of iterative cycle data. An embodiment of iterative cycle data is shown in fig. 3.
The information of the subject personnel is sequentially analyzed according to the sequence of the main keys, so that a label is conveniently set for the next starting, and an object for performing intimacy degree verification with the subject personnel is generated by data smaller than the main keys of the subject personnel.
As shown in table 7, when two pieces of information of the same person are stored in the dynamic track library 330, the principle of the A, B position is that the certificate number is ranked a big way and B small way; the purpose is to discharge the cross dislocation of two people and generate repeated data.
And the data statistical analysis module 240 is configured to perform classification statistics according to the result of system data integration, calculate a relationship intimacy degree score between two people according to the relationship intimacy level, and generate a peer report. The peer report structure is shown as 9, and comprises: the unique identification code, the name of the person A, the certificate number of the person A, the gender of the person A, the birth date of the person A, the name of the person B, the certificate number of the person B, the gender of the person B, the birth date of the person B, the intimacy level A, the intimacy level B, the intimacy level C, the intimacy level D, the relationship intimacy score and the creation time.
Table 9 colleague report form
| Name of field | Data type | Properties | Description of the invention |
| Unique identification code | Numerical type | Main key | |
| Character A name | Character type | | |
| Figure A certificate number | Character type | | |
| Sex of person A | Character type | | |
| Date of birth of character A | Date type | | |
| Character B name | Character type | | |
| Figure B certificate number | Character type | | |
| Sex of person B | Character type | | |
| Date of birth of human B | Date type | | |
| Intimacy rating A | Numerical type | | Number of intimacy degree A |
| Intimacy grade B | Numerical type | | Number of intimacy degree B |
| Intimacy rating C | Numerical type | | Number of intimacy degree C |
| Intimacy rating D | Numerical type | | Number of intimacy degree D |
| Relationship affinity score | Numerical type | | |
| Creation time | Date type | | Default SYS date type |
In a preferred embodiment, the data statistics analysis module 240 includes:
the gathering and counting submodule 241 performs incremental counting to generate a peer report;
the integral operation sub-module 242 calculates the relationship intimacy degree score of the two characters through a relationship intimacy degree score formula based on the occurrence frequency of various intimacy grades in the detail data of the same person related to each event type;
relationship affinity score formula:
n + Trunc (B: N/3,1) + Trunc (C: N/5,1) + Trunc (D: N/10,1), where N represents the number of occurrences and A: N represents the number of occurrences of the intimacy level A class.
And an analysis job monitoring submodule 243 for tracking the operation status of each sub-part in the data statistical analysis module 240.
In a preferred embodiment, the data extraction sub-module 231 may be automatically operated by default data extraction time in the system, and correspondingly, the summary statistics sub-module 241 may also be automatically operated by default data extraction time in the system.
In a preferred embodiment, when the integral operation submodule 242 operates, the intimacy degree information of any two people is calculated by means of iteration cycle data, and a relationship intimacy degree score is obtained.
The business flow chart of the data statistical analysis module 240 is shown in fig. 4:
the gathering and counting submodule 241 automatically operates according to the information in the system dictionary and the information in the detailed data table of the peer, performs incremental counting, and generates a peer report; the integral operation sub-module 242 calculates and records the relationship affinity scores of the two characters in the peer report, and stores the peer report;
if the data of the two same-pedestrians exist in the same-pedestrian report, updating the data, and if the data of the two same-pedestrians do not exist in the same-pedestrian report, increasing the data;
and recording the maximum primary key value of the processed data after each batch of operation is finished, namely the maximum value of the processed data, and storing the maximum value into a system dictionary.
As shown in fig. 1 and 5, the data system 300 includes:
and the business database 310 is used for storing the social basic information. The business database 310 is used for storing personnel hotel accommodation information, personnel civil aviation booking information, personnel entry and exit information and personnel railway transportation information;
the system configuration library 320 is used for storing data information generated in the system, and comprises a data source table, an access data table and an access table field model mapping table;
the dynamic track library 330 is used for storing a detailed data table of the co-workers generated in the data integration process, a report of the co-workers statistically generated by the data statistical analysis module 240, rules of the co-workers, a system log and a system dictionary.
As shown in fig. 1, the presentation system 100 includes:
and a character relationship display module 110, configured to display the character relationship in a form of a table or a graph through the search system analysis result by using the inherent attribute information of the character as a search entry. Wherein the inherent attribute information such as certificate information, name, etc. of the person can be used to uniquely identify the person.
Another aspect of the present invention is to provide an analysis method for mining a potential relationship of a human being based on social basic information, as shown in fig. 6, the method comprising the steps of:
step 1), configuring and receiving data table information of a service database related to social basic information, wherein the data table information comprises establishing database connection for accessing a remote database to form a data source table, and configuring information of an access data table to form an access data table;
the information required to configure the access data table includes:
(i) specifying an existing data source;
(ii) selecting a data table name under a data source;
(iii) adding a data table annotation to the selected data table;
(iv) assigning a data table unique identification code field;
(v) appointing the event type to which the data sheet belongs (the event type comprises guest accommodation, departure and entry, civil aviation ticket booking and railway transportation);
(vi) specifying a data increment field.
Step 2), accessing and mapping the data table, and performing attribute field mapping on the data table in the service database and a predefined data model; the fields in the data table in the service database are associated with the model fields in the predefined data model through attribute field mapping;
step 3), automatically establishing a conversion view for converting the accessed data table into a specified structure in a configuration library according to the information of the data table access mapping, and storing the name of the conversion view into the accessed data table;
and 4), receiving data of the data table in the extracted service database, extracting and transmitting the data values of the attribute field to the dynamic track database 330 according to the mapping condition of the data table access, and reconstructing and storing the data through a set data processing program to acquire the detailed information of the same person.
Specifically, step 4) includes the following substeps:
step 4.1), starting an extraction program of a data table in a service database; the data extraction submodule can automatically run by setting data extraction time in the system;
step 4.2), according to the mapping rule of the access table field model mapping table, transmitting the data value of the mapped service library to the dynamic track library to generate detailed information of the same person;
step 4.3), data is reproduced and stored through a set data processing program; the set data processing program is used for verifying the intimacy degree of corresponding field information of any two persons in the same event type data table according to set peer rules and performing level identification on the relationship intimacy degree of the two persons in the same event;
step 4.4), recording the operation conditions generated in the data extraction, loading and conversion processes each time to form a log record;
step 4.5), the updating condition of the data of the system every day is displayed through the log record generated in the data integration;
and 4.6) tracking the running condition of the data integration module through log records generated in the data integration, and alarming in a message frame mode when abnormity occurs.
And 5) carrying out classified statistics according to the system data integration result, calculating the relationship intimacy degree value between the two characters according to the relationship intimacy grade, and generating a co-pedestrian report.
Specifically, step 5) includes the following substeps:
step 5.1), carrying out incremental statistics to generate a peer report;
step 5.2), calculating the relationship intimacy score of the two characters through a relationship intimacy value formula based on the occurrence frequency of each intimacy grade;
relational intimacy analysis value formula:
n + Trunc (B: N/3,1) + Trunc (C: N/5,1) + Trunc (D: N/10,1), where N represents the number of occurrences and A: N represents the number of occurrences of the intimacy level A class.
And 5.3) tracking the operation condition of each sub-part in the data statistical analysis module.
And 6) taking the inherent attribute information as a retrieval entry, and displaying the human-object relationship in a form of a table or a graph through the analysis result of the retrieval system.