Disclosure of Invention
The invention provides a mobile terminal digital twin model construction method and system for overcoming the technical problems.
In order to achieve the above object, the technical scheme of the present invention is as follows:
a mobile terminal digital twin model construction method comprises the following steps:
S1, acquiring and acquiring non-relational multi-source heterogeneous mobile terminal data;
The multi-source heterogeneous mobile terminal data comprises mobile terminal application data, hardware sensor data and user private data;
S2, constructing a three-dimensional memory mapping relation model;
Data storage is carried out on the multi-source heterogeneous mobile terminal data based on the three-dimensional memory mapping relation model, a three-dimensional memory mapping relation table for storing non-relation data is obtained, field names of the multi-source heterogeneous mobile terminal data are redefined into field indexes corresponding to data fields of the non-relation mobile terminal through the three-dimensional memory mapping table;
carrying out vectorization processing on the multi-source heterogeneous mobile terminal data acquired through the field index, and acquiring vectorized data to store and acquire a mobile terminal database;
S3, providing data retrieval service for the mobile terminal database to realize data retrieval operation of vectorized data in the mobile terminal database;
s4, constructing a virtual entity mapping space according to the vectorized data obtained by the data retrieval operation;
And defining an application agent applied to the virtual entity mapping space, and acquiring a mobile terminal digital twin model for realizing the execution of RPA (remote procedure association) by the real-time sensing processing of the user environment according to the virtual entity mapping space.
Further, the step S2 specifically includes the following steps:
S21, constructing a three-dimensional memory mapping relation model, which comprises a data table setting module, a field set acquisition module, a data type assignment module and a three-dimensional memory mapping relation module;
the data table setting module is used for setting a data table for data storage of the multi-source heterogeneous mobile terminal data according to the set SQLite database;
The field set acquisition module is used for adding data fields to the multi-source heterogeneous mobile terminal data stored in the data table to acquire a data field set F;
And f= { data_0, data_1,..data_n }, wherein data_n represents the n-th data field added;
the data type designating module is used for setting a type label set of each data field type in the data field set;
The three-dimensional memory mapping relation module is used for constructing a three-dimensional memory mapping relation according to the data field set and the type tag set, and acquiring a three-dimensional memory mapping table through the three-dimensional memory mapping relation;
The three-dimensional memory mapping relation is M, F×T type×N- & gt V, wherein M represents a shorthand form of M (F, T, N), T type represents a type tag set, N represents a unique identifier of a data record, V represents a data value containing all multi-source heterogeneous mobile terminal data to be stored, F represents a data field and F epsilon F, T represents a data type and T epsilon T type, and N represents a value corresponding to the record identifier and N epsilon N;
s22, redefining field names of the multi-source heterogeneous mobile terminal data into field indexes corresponding to the data fields of the non-relational mobile terminal through the acquired three-dimensional memory mapping table;
S23, based on the pre-trained continuous word bag model CBOW, carrying out vectorization processing on the multi-source heterogeneous mobile terminal data acquired through the field index, and acquiring vectorized data to store and acquire a mobile terminal database.
Further, the data retrieval service in S3 includes a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval;
the data retrieval service retrieval method based on semantic retrieval comprises the following steps:
constructing a plurality of data vector index navigation layer nodes for data retrieval through HNSW graph algorithm based on a mobile terminal database;
Acquiring edge weights of the data vector index navigation layer nodes based on a heuristic search method;
according to the edge weight and the data vector index navigation layer node, acquiring an index navigation chart structure related to data retrieval;
An embedded Vector search engine J Vector based on single instruction multiple data stream SIMD realizes the data index of a mobile terminal database according to an index navigation chart structure;
The searching method of the data searching service based on SQL searching comprises the following steps:
semantic analysis is carried out on the data of the mobile terminal database through a large language model LLM, and a structured query intention representation is obtained;
based on a pre-trained Text2SQL model, acquiring an SQL sentence according to the structured query intention representation;
And acquiring the SQL query statement based on the SQL statement according to the storage format of the local database and the field index characteristic of the vectorized data through the local data storage mode technology so as to realize the data index of the mobile terminal database according to the SQL query statement.
Further, the step S4 specifically includes the following steps:
S41, customizing an entity model and an entity event of any mobile terminal;
the method comprises the steps of constructing a promt text for acquiring a structural semantic description framework in a self-defined mode according to a physical model and a physical event of a mobile terminal through a natural language technology;
s42, invoking AIGC models of the meristematic diagram/meristematic video, and acquiring virtual digital entity model characterization according to the Prompt text;
S43, converting the Prompt text into a vector to be queried, and retrieving and acquiring multi-source heterogeneous mobile terminal data related to virtual digital entity model representation through the vector to be queried based on a data retrieval service provided by a mobile terminal database;
the method comprises the steps of combining a pre-trained large language model LLM, and acquiring an overview report of an entity event according to the retrieved and acquired multi-source heterogeneous mobile terminal data, wherein the overview report of the entity event is a virtual entity mapping space where a user is located;
S44, defining an application agent applied to the virtual entity mapping space, and determining an execution action instruction of a certain scene in the current user environment based on the overview report of the entity event retrieved and acquired by the virtual entity mapping space;
S45, executing the actions of the mobile terminal according to the executing action instructions so as to realize the function of executing RPA by the user environment real-time sensing processing, and further realize the construction of a digital twin model of the mobile terminal;
And the execution action instruction at least comprises an operation instruction for the interface of the user mobile terminal and an access instruction for the interface of the remote server terminal.
Further, the method for acquiring and acquiring the application data of the mobile terminal in S1 comprises an active acquisition method and a passive acquisition method of the application data based on barrier-free service;
the application data active acquisition method comprises the following steps:
S001, starting authority of barrier-free service of the android system:
enabling the permission to execute gesture operation permission setting for performing simulated click operation on the mobile terminal;
Enabling the permission setting of traversing window content for acquiring interface data;
S002, opening a suspension window for collecting application data of the mobile terminal;
S003, a background server is called to download an application data acquisition plug-in so as to acquire and acquire structured data of application data of the mobile terminal according to the position of the floating window based on simulated click operation;
the application data passive acquisition method comprises the following steps:
s100, starting authority of barrier-free service of an android system:
enabling the permission to execute gesture operation permission setting for performing simulated click operation on the mobile terminal;
Enabling the permission setting of traversing window content for acquiring interface data;
S101, monitoring the change of a mobile terminal page in real time by calling an obstacle-free service monitoring event tool, and when the page is changed, acquiring the structured data displayed on the mobile terminal page in real time by calling an interface active window node acquisition tool, wherein the acquisition action is not executed;
The method for calling the barrier-free service monitoring event tool to monitor the page change of the mobile terminal in real time comprises the following steps:
s1011, acquiring a clicking time when the mobile terminal receives an interface clicking event, and taking the clicking time as a first clicking time;
Acquiring a second clicking moment when the mobile terminal receives the interface clicking event, wherein the second clicking moment is the clicking moment corresponding to the interface clicking event after the interface clicking event corresponding to the first clicking moment;
s1012, acquiring a click time interval between the second click time and the first click time;
judging the size between the clicking time interval and a preset time interval threshold;
And if the clicking time interval is smaller than or equal to the preset time interval threshold, confirming that the page of the mobile terminal is unchanged.
Further, the S1 further includes a method for performing data compression on the multi-source heterogeneous mobile terminal data:
coding information source symbols in the multi-source heterogeneous mobile terminal data based on Huffman coding to obtain information source coding symbols;
Acquiring the occurrence probability of each source coding symbol, and carrying out data compression on multi-source heterogeneous mobile terminal data according to the source coding symbols through a probability statistical model based on the source;
and the expression of the probability statistical model based on the information source is,
Wherein: Representing a desired length of data compression; Representing the average code length; E represents the expected; representing the coding of symbols for all possible sourcesThe Xie represents the set of source code symbols; representing source coded symbolsProbability of occurrence; representing source coded symbolsCorresponding code length.
The mobile terminal digital twin model construction system comprises a data acquisition module, a data compression module, a three-dimensional memory mapping relation model construction module, a data vectorization module, a data retrieval service module, an application agent setting module and a mobile terminal digital twin model construction module;
the data acquisition module is used for acquiring and acquiring non-relational multi-source heterogeneous mobile terminal data;
The multi-source heterogeneous mobile terminal data comprises mobile terminal application data, hardware sensor data and user data;
the data compression module is used for carrying out data compression on the multi-source heterogeneous mobile terminal data to obtain compressed data;
The three-dimensional memory mapping relation model construction module is used for constructing a three-dimensional memory mapping relation model;
Data storage is carried out on the multi-source heterogeneous mobile terminal data based on the three-dimensional memory mapping relation model, a three-dimensional memory mapping relation table for storing non-relation data is obtained, field names of the multi-source heterogeneous mobile terminal data are redefined into field indexes corresponding to data fields of the non-relation mobile terminal through the three-dimensional memory mapping table;
The data vectorization module is used for vectorizing the multi-source heterogeneous mobile terminal data acquired through the field indexes to acquire vectorized data for storing and acquiring a mobile terminal database;
The data retrieval service module is used for providing data retrieval service for the mobile terminal database so as to realize the data retrieval operation of the vectorized data in the mobile terminal database;
And the data retrieval service comprises a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval;
The application agent setting module is used for setting application agents applied to the virtual entity mapping space, and the application agents at least comprise an information security agent and a health management agent;
The mobile terminal digital twin model construction module is used for constructing a virtual entity mapping space according to the vectorized data acquired by the data retrieval operation, and acquiring a mobile terminal digital twin model for realizing the user environment real-time perception processing execution RPA according to the virtual entity mapping space based on the application agent.
The method and the system have the advantages that the method and the system for constructing the digital twin model of the mobile terminal effectively realize the acquisition and integration of the data fragment information of the non-relational multi-source heterogeneous mobile terminal by carrying out data storage on the multi-source heterogeneous mobile terminal data based on the constructed three-dimensional memory mapping relation model, acquire the three-dimensional memory mapping relation table for storing the non-relational data, greatly improve the comprehensiveness of acquiring the digits of the user mobile terminal, redefine the field names of the multi-source heterogeneous mobile terminal data into field indexes corresponding to the data fields of the non-relational mobile terminal through the three-dimensional memory mapping table, carry out vectorization processing on the multi-source heterogeneous mobile terminal data acquired through the field indexes, acquire vectorized data to store and acquire a mobile terminal database, effectively realize the structured storage of the multi-source heterogeneous mobile terminal data, conveniently provide complete data export service for users, lay a foundation by defining an application intelligent body applied to a virtual entity mapping space, acquire a mobile terminal digital twin model for realizing the implementation of real-time sensing and RPA (RPA) on the basis of realizing the real-time sensing processing of a user environment according to the constructed virtual entity mapping space, and provide great knowledge of the user on the aspects of managing and maintenance of own data, use data, service data and application and data processing conditions.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The embodiment provides a mobile terminal digital twin model construction method, as shown in fig. 1, comprising the following steps:
S1, acquiring and acquiring non-relational multi-source heterogeneous mobile terminal data;
The multi-source heterogeneous mobile terminal data comprises mobile terminal application data, hardware sensor data and user private data;
The intelligent mobile phone is characterized in that the intelligent mobile phone is used as a personal daily data acquisition terminal as the function of the intelligent mobile phone is continuously enhanced, meanwhile, the acquired all user data is required to be locally stored on the mobile terminal to be completely covered by combining with the prior art, and the personal data, namely the mobile terminal data, is divided into three main types, namely application data, data generated by the user using application services provided by various application manufacturers, sensor data based on data acquired by sensors and communication modules inside and outside the intelligent terminal, and data of ideas, records, behaviors and the like generated by the user at any time;
The application data is distributed among various large application manufacturers, the data export function is not provided, and the application data cannot be acquired in a traditional data web crawler mode due to different technical frameworks used by various system applications, so that acquiring and integrating application related data becomes extremely difficult;
The embodiment collects application data on the mobile intelligent terminal device in a manner that the user sees the application data, the collection manner of the application data comprises an application data active collection method and an application data passive collection method based on barrier-free service (Accessibility Service), the active collection is personal data collection of third party application, and the application data active collection method specifically comprises the following steps:
S001, starting the authority of barrier-free service of an Android (Android) system:
Comprises enabling the permission (can Perform Gestures) for executing gesture operation to set up the simulated click operation for the mobile terminal;
Enabling a traversal-allowed Window Content authority (CAN RETRIEVE Window Content) setting for acquiring interface data;
S002, starting a floating window for collecting application data of the mobile terminal, namely a movable window;
S003, a background server is called to download an application data acquisition plug-in so as to acquire and acquire structured data of application data of the mobile terminal according to the position of the floating window based on simulated click operation;
The method specifically comprises the following steps:
S0031, starting a third party APP according to a set third party APP package name;
S0032, acquiring a root node of an accessible tree in a current active window through an accessible service interface active window node acquisition tool (get Root In Active Window) so as to acquire the similarity between the current active window node tree and an expected setting interface node tree of a final data interface of a third-party APP, and further determining whether the current active window interface belongs to the expected setting interface of the final data interface of the third-party APP;
The method for obtaining the similarity between the current active window node tree and the expected setting interface node tree of the final data interface of the third party APP is a prior known technical means and is not repeated herein;
s0033, if the current active window interface belongs to the expected setting interface of the final data interface of the third party APP of the mobile terminal, the next-stage functional page of the current active window is the final data interface;
If the current active window interface does not belong to the expected setting interface of the final data interface of the third party APP of the mobile terminal, entering a next-stage functional page of the current active window, and repeatedly executing S132 until the current active window interface belongs to the expected setting interface of the final data interface of the third party APP of the mobile terminal, and acquiring a final data interface;
S0034, if the data of the final data interface are displayed in a list or loaded in real time through a network, executing S0035;
If the final data interface is presented in a picture form, executing S0036;
S0035, performing downward sliding page turning on a final data interface until no new data is added or acquisition requirements are met, traversing all node elements of the final data interface according to a method of an interface active window node acquisition tool (get Root In Active Window) of barrier-free service, and acquiring large character strings of all text data of the final data interface;
S0036, storing the pictures to a local or remote server, and splicing the picture paths to form large character strings of all the picture paths of the final data interface;
s0037, obtaining final data interface structured data of the third party APP of the mobile terminal based on the regular expression according to the large character string.
In this embodiment, the application data is passively collected, that is, some information seen by the user when using the APP, including data received by talking chat, browsing news, online shopping, brushing a circle of friends, and the like, and the latest received knowledge information of the user can be effectively known by collecting the data that the user views in real time, so as to identify the recent state of the user, and the like. The traditional mode used for collecting third party application data is OCR (optical character recognition) after screen capturing, but because of the performance problem of a mobile phone, the information seen by a user cannot be completely collected in real time, so the application provides an application data passive collection method based on barrier-free service (Accessibility Service), and the application data passive collection method specifically comprises the following steps:
S100, starting the authority of barrier-free service of an Android system, wherein the mobile terminal is a mobile phone of the Android system;
Comprises enabling the permission (can Perform Gestures) for executing gesture operation to set up the simulated click operation for the mobile terminal;
Enabling a traversal-allowed Window Content authority (CAN RETRIEVE Window Content) setting for acquiring interface data;
S101, monitoring the change of a mobile terminal page in real time by calling an obstacle-free service monitoring event tool (on Accessibility Event), and when the page is changed, acquiring the structured data displayed on the mobile terminal page in real time by calling an interface active window node acquisition tool (get Root In Active Window);
the method for calling the barrier-free service monitoring event tool (on Accessibility Event) to monitor the page change of the mobile terminal in real time comprises the following steps:
s1011, acquiring a clicking time when the mobile terminal receives an interface clicking event, and taking the clicking time as a first clicking time;
Acquiring a second clicking moment when the mobile terminal receives the interface clicking event, wherein the second clicking moment is the clicking moment corresponding to the interface clicking event after the interface clicking event corresponding to the first clicking moment;
s1012, acquiring a click time interval between the second click time and the first click time;
judging the size between the clicking time interval and a preset time interval threshold;
If the clicking time interval is smaller than or equal to the preset time interval threshold, confirming that the page of the mobile terminal is unchanged;
The hardware sensor data can well detect and identify the physical environment where the user is located, the NFC tag can finely transmit the real-time scene of the user to the set digital twin space, the physical space position where the user is located can be judged through WIFI and Bluetooth connection data, the data processing algorithm can identify the user-defined scene of the data such as GPS, gyroscope, gravity sensing and light sensing, and the state acquisition of the physical world where the user is located can be enriched by combining an external intelligent hardware sensor, so that intelligent help more fitting the physical state of the user can be made in the digital twin space.
The user private data is provided, namely, all private data in life can be recorded by a user through text, voice, photo and video, the user can be helped to digitize the previously unconscious information by collecting the data through a convenient and quick information recording mode, a digital twin space can be constructed to visualize the data by using an artificial intelligent large model technology, and solid and powerful basic data is provided for personal digital twin;
In a specific embodiment, the method for performing data compression on the multi-source heterogeneous mobile terminal data is further included in S1:
coding information source symbols in the multi-source heterogeneous mobile terminal data based on Huffman coding to obtain information source coding symbols;
Acquiring the occurrence probability of each source coding symbol, and carrying out data compression on multi-source heterogeneous mobile terminal data according to the source coding symbols through a probability statistical model based on the source;
The problem that the local Data processing speed cannot keep pace with the acquisition speed happens in the actual Data acquisition process is solved, vector Data retrieval is carried out on the user attention dimensional Data, namely, the acquired mobile terminal Data are processed by carrying out Data compression on the application scene corresponding to the user attention program, namely, the user setting through the back end recognition matching, according to the priority corresponding to the Data type, the Data are transmitted to a large model at fixed time, and the fact that the same information Source symbol content characteristic frequently occurs in the daily Data of the user is considered in the embodiment, wherein the Data information Source (Data Source) of the daily Data of the user is a core concept in the fields of Data science, information technology and communication, and the Data Source refers to the original Source of the Data or an entity/system for providing the Data. The method can be any medium, equipment, platform or environment for generating, storing or transmitting data, and the quality, reliability and type of a data information source directly influence the application value and analysis result of the data;
therefore, the embodiment adopts Huffman coding and defines probability according to the occurrence frequency of the signal source symbol in the data so as to realize the aim of data compression according to a signal source-based probability statistical model;
And the expression of the probability statistical model based on the information source is,
Wherein: Representing a desired length of data compression; Representing the average code length; E represents the expected; representing the coding of symbols for all possible sourcesThe Xie represents the set of source code symbols; representing source coded symbolsProbability of occurrence; Representing source symbolsThe source code symbol with large probability codes short and the source code symbol with small probability codes long;
S2, constructing a three-dimensional memory mapping relation model;
Data storage is carried out on the multi-source heterogeneous mobile terminal data based on the three-dimensional memory mapping relation model, a three-dimensional memory mapping relation table for storing non-relation data is obtained, field names of the multi-source heterogeneous mobile terminal data are redefined into field indexes corresponding to data fields of the non-relation mobile terminal through the three-dimensional memory mapping table;
carrying out vectorization processing on the multi-source heterogeneous mobile terminal data acquired through the field index, and acquiring vectorized data to store and acquire a mobile terminal database;
The application of the standard digital twin technique in this embodiment mainly focuses on the accurate one-to-one replication modeling of target objects in the physical world using high-precision three-dimensional models. Extensive data acquisition work is involved, i.e., data is densely collected from each key location point of a physical target entity to ensure that a physical map can be constructed in virtual digital space that is highly consistent with and complete to the physical object. The complexity and diversity of the physical world is such that model data exhibits multi-dimensional characteristics. In particular, for a complex-structure, functionally diverse, data-driven entity, modeling by considering only its basic morphology is far from sufficient. The requirements of users for digital twin applications also often have a high degree of individuation, and models in the virtual space need to be flexibly customized according to the actual requirements of the users. In view of this, the embodiment is based on a comprehensive data acquisition technology, and performs data processing and integration on various generalized data of a user in daily use by constructing a three-dimensional memory mapping relationship model to obtain comprehensive information about user preferences, behavior habits and potential requirements;
the acquired data has very wide sources including active and passive, real-time and historical, full and incremental dimensions, various data types, including structured, unstructured and semi-structured data, and the data specifically comprises various forms such as text, voice, pictures and video, so that in order to better construct a digital twin space model based on the data, an effective strategy needs to be adopted to preprocess the data, the data is conventionally structured according to a standard and then put into storage, but the acquired data sources are complicated, so that the integrity and expansibility of some data are lost, the user related private data cannot participate in the construction of a model due to dimensional uncertainty, and meanwhile, the original data are required to be stored, the multi-source heterogeneous data are required to be formatted and stored, the existing scheme can realize dynamic increase and decrease of fields and data tables through the non-relational data, but the selection of the non-relational data base is relatively limited under the condition that the calculation performance of a mobile terminal is weak, and the situation that the content of the fields can be understood is difficult to understand when the data is used, so that the three-dimensional heterogeneous data is mapped to realize the construction of the three-dimensional heterogeneous data through the mobile terminal;
in a specific embodiment, the step S2 specifically includes the following steps:
S21, constructing a three-dimensional memory mapping relation model, which comprises a data table setting module, a field set acquisition module, a data type assignment module and a three-dimensional memory mapping relation module;
the data table setting module is used for setting a data table for data storage of the multi-source heterogeneous mobile terminal data according to the set SQLite database;
The field set acquisition module is used for adding data fields to the multi-source heterogeneous mobile terminal data stored in the data table to acquire a data field set F;
And f= { data_0, data_1,..data_n }, wherein data_n represents the n-th data field added;
the data type designating module is used for setting a type label set of each data field type in the data field set;
The three-dimensional memory mapping relation module is used for constructing a three-dimensional memory mapping relation according to the data field set and the type tag set, and acquiring a three-dimensional memory mapping table through the three-dimensional memory mapping relation;
The three-dimensional memory mapping relation is M, F and T type is N and V, wherein M represents a shorthand form of M (F, T, N), T type represents a type tag set, N represents a unique identifier of a data record, V represents a data value containing all multi-source heterogeneous mobile terminal data to be stored, F represents a data field, F is E F, T represents a data type, T is E T type, N represents a value corresponding to the record identifier, N is E N, the model can store data from different sources due to the flexibility of F and T type, the data can be heterogeneous and can support standard SQL query data, the embodiment can easily query, update and maintain the relation among the data type fields through M mapping, pre-processing is carried out when the standard SQL data query is used, and the field names are redefined as field indexes corresponding to the fields through a three-dimensional memory mapping model, so that the potential value of the data can be fully exerted in the use process of the subsequent data;
s22, redefining field names of the multi-source heterogeneous mobile terminal data into field indexes corresponding to the data fields of the non-relational mobile terminal through the acquired three-dimensional memory mapping table;
S23, based on a pre-trained continuous word bag model CBOW, carrying out vectorization processing on multi-source heterogeneous mobile terminal data acquired through field indexes, and acquiring vectorized data to store and acquire a mobile terminal database;
In particular, in the embodiment, text data acquired by a mobile terminal is converted into vectors and original data mapping storage by combining a Word2vec with a self-defined dictionary and a Word vector mode, other unstructured or semi-structured data slices are converted into vectors and original data mapping storage by using TIKA tools, when the Word2vec is used, the problem of insufficient computing performance of the mobile terminal is considered, a Word2vec technology improved by Embedding is introduced to vectorize CBOW (Continuous Bag-of-Words Model) Continuous Word Bag Model pre-trained by using related data, specific lines are selected from a weight matrix directly to be transmitted to the next layer, lengthy matrix multiplication operation is avoided, and the Embedding layer is used for extracting Word ID corresponding lines (vectors) from the weight parameters, so that the computing amount is reduced, and meanwhile, the usability and practicability of data in subsequent modeling are greatly improved by combining with the common dictionary in the life and working of users;
According to the embodiment, non-relational data is stored by using a relational database SQLite which is more suitable for being embedded in a mobile terminal, a data table is created, data fields data_0 to data_63 are added, the data types of 64 fields are supported at most, the data types are specified by adding data_type fields, a three-dimensional memory mapping relation model is built to index original fields, multi-source heterogeneous data which is similar to the characteristics of a columnar database can be better stored, the relation among various data type fields is better maintained, and the method is more suitable for an OLAP (on-line analysis processing) scene, the expansibility of the data types and the fields is enhanced, and in addition, formatted storage can be realized by customizing the data format of any data type through a preset data system or a user;
S3, providing data retrieval service for the mobile terminal database to realize data retrieval operation of vectorized data in the mobile terminal database;
in a specific embodiment, the data retrieval service provided for the mobile terminal database comprises a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval;
Specifically, the traditional keyword matching algorithm only considers the similarity in terms of vocabulary, but ignores the relation between sentence structure and semantics, the semantic search algorithm can better understand the query intention of the user, and provides more accurate search results, and in order to accelerate the Vector search speed, the embodiment performs local search on related processed mobile terminal Data by using an embedded Vector search engine J Vector accelerated by a graph algorithm and SIMD (Single Instruction, multiple Data);
the data retrieval service retrieval method based on semantic retrieval comprises the following steps:
constructing a plurality of data vector index navigation layer nodes for data retrieval through HNSW graph algorithm based on a mobile terminal database;
Acquiring edge weights of the data vector index navigation layer nodes based on a heuristic search method;
according to the edge weight and the data vector index navigation layer node, acquiring an index navigation chart structure related to data retrieval;
An embedded Vector search engine J Vector based on single instruction multiple data stream SIMD realizes the data index of a mobile terminal database according to an index navigation chart structure;
specifically, the embodiment can automatically analyze the user problem through a large language model and Text2SQL and combining a local optimized data storage mode technology, and convert the user problem into a corresponding SQL query statement, so that the flow of database query is simplified, the accuracy and efficiency of the query are improved, the threshold of database operation is reduced, more users can conveniently acquire and utilize structured data, and complex database query work can be completed without professional SQL knowledge;
The searching method of the data searching service based on SQL searching comprises the following steps:
semantic analysis is carried out on the data of the mobile terminal database through a large language model LLM, and a structured query intention representation is obtained;
based on a pre-trained Text2SQL model, acquiring an SQL sentence according to the structured query intention representation;
Acquiring SQL query sentences based on the SQL query sentences according to the storage format of a local database and field index features of vectorized data through a local data storage mode technology so as to realize data index of the mobile terminal database according to the SQL query sentences;
s4, constructing a virtual entity mapping space according to the vectorized data obtained by the data retrieval operation;
Defining an application agent applied to a virtual entity mapping space, and acquiring a mobile terminal digital twin model for realizing the execution of RPA (remote procedure association) by the real-time sensing processing of a user environment according to the virtual entity mapping space;
In the process of constructing a virtual model of a physical entity applied to a digital twin space, the conventional method generally uses Unity 3D/Unreal Engine to perform pre-modeling, and acquires relevant data of the physical entity in real time to realize comprehensive synchronous mapping with the virtual entity. However, when modeling a digital twin space of a related entity of daily work and life of a user under a mobile terminal, the user can only select a system to perform digital twin construction on a physical entity which is modeled in advance, and can only display information of a preset foundation of the system, and the digital twin technology can not be applied to various aspects of life of the user to adapt to personalized requirements of the user;
The method specifically comprises the following steps:
S41, customizing an entity model and an entity event of any mobile terminal;
the method comprises the steps of constructing a promt text for acquiring a structural semantic description framework in a self-defined mode according to a physical model and a physical event of a mobile terminal through a natural language technology;
s42, invoking AIGC models of the meristematic diagram/meristematic video, and acquiring virtual digital entity model characterization according to the Prompt text;
S43, converting the Prompt text into a vector to be queried, and retrieving and acquiring multi-source heterogeneous mobile terminal data related to virtual digital entity model representation through the vector to be queried based on a data retrieval service provided by a mobile terminal database;
the method comprises the steps of combining a pre-trained large language model LLM, and acquiring an overview report of an entity event according to the retrieved and acquired multi-source heterogeneous mobile terminal data, wherein the overview report of the entity event is a virtual entity mapping space where a user is located;
s44, defining application agents applied to the virtual entity mapping space, such as an information security agent and a health management agent, and determining an execution action instruction of a certain scene in the current user environment based on the overview report of the entity event retrieved and acquired by the virtual entity mapping space;
S45, executing the action of the mobile terminal according to the executing action instruction so as to realize the function of executing RPA by the user environment real-time sensing processing, and further realize the construction of a digital twin model of the mobile terminal, wherein the executing action instruction at least comprises an operation instruction for a user mobile terminal interface and an access instruction for a remote server interface. According to the embodiment, through Accessibility Service barrier-free services, operations such as clicking, sliding and inputting of an interface can be executed to realize an RPA function, action recording is provided for a user to realize more convenient operation, a transparent suspension window is started to record a click coordinate path of the user, recording of the user action is performed, and linkage can be realized by associating the action with an application scene, for example, a digital twin space recognizes that a scene is connected to home and automatically connected with WIFI, the current environment temperature is not suitable, the action is automatically found out from Mijia application and then an air conditioner is opened, the problem of traditional rule-based scene automation is solved through large model scene recognition, and the linkage limitation among various applications is broken;
According to the embodiment, AIGC technology is adopted, a user is allowed to customize a physical model or object dimension of a mobile terminal, a Prompt is built through user natural language customization, a virtual digital entity is built through a text-generated graph/text-generated video, meanwhile, all relevant original data of the description entity are obtained through semantic retrieval of collected vectorized relevant data through the custom Prompt natural language, RAG (RETRIEVAL-augmented Generation) complete functions of local data of the mobile terminal are realized by combining a large language model LLM, the retrieved and matched data are processed through the large model to obtain an overview report of the digital entity, in addition, the user can redefine the Prompt, further process and return dimension information concerned by the user through the large model, and a placeholder is provided for real-time dimension information to be embedded into a large model answer result, so that high-rate information synchronization can be kept, and meanwhile, the user can further see and export detailed structured data related to the entity model through the provided data retrieval function;
The embodiment also comprises a mobile terminal digital twin model construction system, as shown in fig. 2, which comprises a data acquisition module, a data compression module, a three-dimensional memory mapping relation model construction module, a data vectorization module, a data retrieval service module, an application agent setting module and a mobile terminal digital twin model construction module;
the data acquisition module is used for acquiring and acquiring non-relational multi-source heterogeneous mobile terminal data;
The multi-source heterogeneous mobile terminal data comprises mobile terminal application data, hardware sensor data and user data;
the data compression module is used for carrying out data compression on the multi-source heterogeneous mobile terminal data to obtain compressed data;
The three-dimensional memory mapping relation model construction module is used for constructing a three-dimensional memory mapping relation model;
Data storage is carried out on the multi-source heterogeneous mobile terminal data based on the three-dimensional memory mapping relation model, a three-dimensional memory mapping relation table for storing non-relation data is obtained, field names of the multi-source heterogeneous mobile terminal data are redefined into field indexes corresponding to data fields of the non-relation mobile terminal through the three-dimensional memory mapping table;
The data vectorization module is used for vectorizing the multi-source heterogeneous mobile terminal data acquired through the field indexes to acquire vectorized data for storing and acquiring a mobile terminal database;
The data retrieval service module is used for providing data retrieval service for the mobile terminal database so as to realize the data retrieval operation of the vectorized data in the mobile terminal database;
And the data retrieval service comprises a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval;
The application agent setting module is used for setting application agents applied to the virtual entity mapping space, and the application agents at least comprise an information security agent and a health management agent;
The mobile terminal digital twin model construction module is used for constructing a virtual entity mapping space according to the vectorized data acquired by the data retrieval operation, and acquiring a mobile terminal digital twin model for realizing the user environment real-time perception processing execution RPA according to the virtual entity mapping space based on the application agent.
The model application of the mobile terminal digital twin model constructed in the embodiment is that through collected various data in daily life of a personal user, the user can customize complete digital twin virtual mapping of various entities in life, has detailed state checking and control on the entities, simultaneously provides application of the user to the whole twin space data, and grasps rich language knowledge and modes through pre-training on large-scale text data based on a large language model. The models can understand complex language structures, generate coherent and strong-logic texts, further enable a large language model to analyze queries or prompts input by users, understand the intention and the requirement behind the queries or prompts, and further conduct accurate analysis and decision. For example, a user needs movie recommendation, a semantic analysis model constructed by a daily relevant corpus of the user is used for carrying out vector retrieval on relevant data such as user preference, state, movies and the like according to a question of the user, submitting the relevant data to a large language model for processing response, and simultaneously providing various data AI AGENT (intelligent AGENT) for deeper mining on user data, for example, an information security intelligent AGENT can give a security report and repair suggestion through detecting all data of the user acquired in real time, and find whether the current behavior of the user is abnormal or not, find whether the user is suffering from fraud and the like through interface text dialogue content acquired in real time, and a health management intelligent AGENT gives accurate analysis based on real complete data to the acquired comprehensive analysis of the daily diet, movement, physiology, sleep and the like of the user.
The method has the beneficial effects that:
According to the embodiment, the three-dimensional memory mapping relation model is constructed to store data of the multi-source heterogeneous mobile terminal, a three-dimensional memory mapping relation table for storing non-relational data is obtained, collection and integration of data fragment information of the non-relational multi-source heterogeneous mobile terminal are effectively achieved, comprehensiveness of obtaining the user mobile terminal is greatly improved, field names of the multi-source heterogeneous mobile terminal data are redefined to field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping relation table, vectorization processing is conducted on the multi-source heterogeneous mobile terminal data obtained through the field indexes, vectorization data are obtained to store and obtain a mobile terminal database, structural storage of the multi-source heterogeneous mobile terminal data is effectively achieved, foundation is laid for providing complete data export service for users conveniently, and a mobile terminal digital twin model for realizing real-time sensing processing of a user environment is obtained according to the constructed virtual entity mapping space, so that the user can greatly improve application technology conditions of managing data safety, maintenance data, processing service data and data of the user terminals and the like.
It should be noted that the above embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solution described in the above embodiments can be modified or some or all of the technical features thereof can be equivalently replaced, and the modification or replacement does not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the present invention.