Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
As one of ordinary skill in the art can know, with the development of technology and the appearance of new scenes, the technical scheme provided by the embodiment of the application is also applicable to similar technical problems.
The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely illustrative of the manner in which embodiments of the application have been described in connection with the description of the objects having the same attributes. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
In order to solve the problem of low file processing efficiency in the prior art, embodiment 1 of the present application provides a file processing system.
Fig. 1 to fig. 3 are a file processing system according to embodiment 1 of the present invention. The file processing system shown in fig. 1 comprises a business system 11 and an electronic archiving system 12.
The service system 11 is configured to send storage event information indicating that the file identifier and the file storage path are related to the electronic filing system 12 after storing the file generated during the running process of the service system itself in a corresponding storage medium.
The storage medium is a physical device or medium for storing and persisting data, and may be an electronic device, a magnetic medium, an optical medium, etc., such as a hard disk drive, a solid state disk, an optical disk, a magnetic tape. Storage of files on storage media is achieved by means of storage techniques, which are used in this proposal, including NAS storage and object storage.
The file identifier is used for distinguishing different files and can be in different forms such as file numbers, single numbers and the like; the file storage path may be a specific location of the file in the storage medium.
The storage event information further includes information such as generation event of the information, document type (such as policy, security policy, core policy, group policy, etc. related in the business system of insurance industry), storage type (or storage technology, such as NAS storage, object storage mentioned above), etc.
In one implementation, business system 11 may send storage event information to a cluster of message queues. A message queue cluster is a clustered system of multiple message queue servers, such as RabbitMQ, kafka, activeMQ open source message queues, that handle large message transmission and processing requirements. In the application, the message queue cluster is used for receiving the storage event information sent by each service system, and the storage event information is rapidly processed through a load balancing mechanism in the cluster.
The electronic filing system 12 is configured to obtain storage event information sent by each service system; determining a server for processing the stored event information according to the idle state of each server in the server cluster, so that the server executes: writing the storage event information into a database; acquiring a downloading request of a target file sent by the service system; polling each server within a server cluster, determining the server that handled the download request, causing the server to perform: determining a file identification of the target file according to the downloading request; acquiring storage event information corresponding to the target file from the database according to the file identification; extracting a target file from a storage medium according to the storage event information, and feeding back an extraction result to a service system which sends the downloading request;
Electronic archiving system 12 may obtain stored event information sent by each business system from the message queue cluster.
An F5 load balancer is deployed within the electronic archiving system 12 to stream storage event information obtained from the message queue cluster onto each server of the server cluster. High availability and high reliability of the system are ensured. And F5, the load balancer determines the server for processing the stored event information according to the idle state of each server in the server cluster, and if the idle state of each server can be determined by adopting a minimum connection number algorithm.
And the database is opened to users for access so that the users can acquire files generated in the running process of each service system based on the stored event information.
The system that directly receives the user download request may be each service system 11 or may be the electronic filing system 12, and two implementations will be described below. The electronic filing system 12 will be described based on the processing of the download request by taking the case where the user makes the download request through the service system 11 as an example.
The load balancer in the electronic archiving system 12 is further configured to poll each server within the server cluster based on the download request and determine the server that handled the download request. The load balancer adopts a polling mechanism, so that the utilization rate of each server can be improved, and long-time waiting of users is avoided. Illustrating: the number of tasks currently being executed by the server 1 is 1, and the execution duration is expected to be 10 minutes; the number of tasks currently being executed by the server 2 is 10, and the expected execution duration is 70 seconds; the load balancer, via the polling mechanism, will determine that the server 2 is acting as the server that handles the download request, avoiding the latency incurred by using the minimum number connection algorithm (if this algorithm is used, determining that the server 1 is the executing server, the user needs to wait a longer time).
The download request includes a file identifier of the target file, the server processing the download request accesses the database, and obtains corresponding storage event information from the database based on the file identifier, where the storage event information includes information that a storage path, a storage type, and the like of the file can obtain the file from the storage medium, so that the target file can be extracted from the storage medium based on the storage event information. Wherein the location of the file in the storage medium is determined based on the storage path; and determining a read-write mode of the file, namely an extraction mode of the target file, based on the storage type.
As shown in fig. 2, the topology of the file processing system is schematically depicted when a user issues a request for downloading a target file via the electronic archiving system 12. A specific workflow of a file processing system comprising:
in step 201, the service system stores the files generated in the respective running process, such as electronic policy, on the storage medium corresponding to the NAS service or the object storage service.
Step 202, after the file is successfully saved, the service system sends a push message, namely the storage event information, to the message queue cluster; the storage event information is used to indicate information about file identifications and file storage paths.
In step 203, the F5 load balancer of the electronic archiving system monitors the storage event information in real time, and determines a server for processing the storage event information based on a minimum connection number algorithm.
In step 204, the determined server stores the stored event information in a write database of the database, and the write database synchronizes the stored event information to a read database.
Thus, based on steps 201 to 204, archiving of the file is completed.
The file processing system is also used for processing the downloading request of the user to the target file. The method specifically comprises the following steps:
Step 211, a load balancer of the file archiving system receives a download request of a user to a target file, and a polling mechanism is adopted to determine a server for processing the download request;
step 212, the server processing the download request determines the storage event information corresponding to the target file based on the file identification in the download request, and requests the storage event information from the write database;
step 213, the write database returns the stored event information to the server processing the download request;
step 214, the server processing the download request requests the target file from the storage medium based on the storage path related information in the storage event information;
Step 215, returning the corresponding storage medium to the target file;
In step 216, the electronic filing system feeds the downloaded target file back to the user.
Thus, the downloading of the file is completed based on steps 211 to 216.
When the electronic filing system downloads the file, the downloaded target file is not limited by the service systems, namely, the file generated by each service system can be downloaded only by writing the corresponding storage event information into the database, so that the file processing efficiency is improved, and great convenience is provided for a user to acquire the file.
As shown in fig. 3, when a user issues a download request of a target file through the service system 11, a topology diagram of the file processing system is shown. A specific workflow of a file processing system comprising:
In step 301, the service system stores the files generated in the respective running process, such as electronic policy, on a storage medium corresponding to the NAS service or the object storage service.
Step 302, after the file is successfully saved, the service system sends a push message, namely the storage event information, to the message queue cluster; the storage event information is used to indicate file identification and file storage path related information.
Step 303, the F5 load balancer of the electronic archiving system monitors the storage event information in real time, and determines a server for processing the storage event information based on the minimum connection number algorithm.
In step 304, the determined server stores the storage event information in a write database of the database, and the write database synchronizes the storage event information to a read database.
Thus, the filing of the file is completed based on steps 301 to 304.
When the file processing system processes a download request for a target file, the file processing system specifically comprises:
Step 311, the user logs in a service system, and based on the service system, sends a download request for the target file, where the download request includes a file identifier of the target file.
In step 312, the service system pushes the download request message to the message queue cluster.
Step 313, the load balancer of the electronic archiving system monitors the download request message sent by the message queue cluster in real time, and determines the server for processing the download request based on the polling mechanism;
step 314, the server processing the download request determines the storage event information corresponding to the target file based on the file identification in the download request, and requests the storage event information from the write database;
step 315, the write database returns the stored event information to the server processing the download request;
step 316, the server processing the download request requests the target file from the storage medium based on the storage path related information in the storage event information;
Step 317, the corresponding storage medium returns the target file to the electronic filing system;
step 318, the electronic filing system feeds back the target file to the service system which sends the downloading request;
in step 319, the service system returns the downloaded target file to the user.
Thus, based on steps 311 to 319, the downloading of the file is completed.
When the service system downloads the file, the downloaded target file is not limited by the service system, namely, the file generated by each service system can be downloaded only by writing the corresponding storage event information into the database, so that the efficiency of file processing is improved, and great convenience is provided for a user to acquire the file.
FIG. 4 is a schematic diagram of a query interface of an electronic archiving system. In an archive information query page of the electronic archive system, a user can query archive information, archive records, download records and other information of documents in real time through file identification, such as a document number.
As shown in FIG. 5, a schematic diagram of a display interface for feedback from the electronic archiving system after the user sets the query conditions. When the archiving information of the file is checked, the information displayed by the interface comprises information such as a single number, file generation time, file production completion time, file warehousing archiving time, time difference (the number of minutes is different) and the like.
As shown in fig. 6, a schematic diagram is provided for an interface of the electronic filing system for watermarking a downloaded file. The setting of adding the watermark to the file can comprise the added watermark text, watermark text format, the number of the watermark of each page and the like.
FIG. 7 is a schematic diagram of an interface for inhibiting downloads in an electronic archiving system. When the file, such as the electronic policy shown in the figure, has a mark for prohibiting downloading when the file is not completely generated or is in error generation, the mark for prohibiting downloading is set for the file, so that the file downloaded by a user is complete and accurate.
FIG. 8 is a schematic diagram of data statistics formed by the electronic archiving system. The statistical data may include file downloading amount, downloading comparability, file number and the like. Fig. 8 is a bar chart showing the number of file download successes within a set period of time (2022-5-12 days).
Example 2
The embodiment 2 of the application provides a file processing method, which is used for solving the problem of low file processing efficiency in the prior art.
The execution subject of the method can be any computing device that can implement the method, such as a server, a mobile phone, a personal computer, an intelligent wearable device, an intelligent robot, and the like.
In addition, the execution sequence of the different steps is not limited in the embodiment of the application. When the method provided by the embodiment of the application is used, the execution sequence of different steps can be adjusted according to actual requirements.
For convenience of description, the method provided by the embodiment of the present application will be described in detail below by taking a device for processing a file as an execution body of the method.
As shown in fig. 9, a file processing method provided in this embodiment specifically includes the following steps:
Step 91, obtaining storage event information sent by each service system; and storing the event information, namely storing the files generated in the respective operation process to the corresponding storage medium for each service system, and sending information for indicating the file identification and the file storage path.
The obtaining the storage event information sent by each service system specifically includes:
Acquiring information sent by each service system from a message queue cluster; and after storing the files generated in the respective running process to the corresponding storage medium, each service system sends the storage event information to the message queue cluster.
The file identifier is used for distinguishing different files and can be in different forms such as file numbers, single numbers and the like; the file storage path may be a specific location of the file in the storage medium.
The storage event information further includes information such as generation event of the information, document type (such as policy, security policy, core policy, group policy, etc. related in the business system of insurance industry), storage type (or storage technology, such as NAS storage, object storage mentioned above), etc.
Step 92, writing the storage event information into a database; and the database is opened to users for access so that the users can acquire files generated in the running process of each service system based on the stored event information.
In one implementation, the storage event information is received by a load balancer, and then the storage event information is written into a database, including;
And the load balancer determines the server for processing the stored event information according to the idle state of each server in the server cluster, so that the server writes the stored event information into the database.
In one implementation, the method further includes processing the file download request, and the specific steps include:
acquiring a downloading request of a target file; the downloading request is sent out through the service system; the download request includes a file identification of the target file.
Determining a file identification of the target file according to the downloading request;
reading storage event information corresponding to the target file from the database according to the file identification;
And extracting the target file from the storage medium based on the storage event information, and feeding back an extraction result to the service system which sends the downloading request.
The obtaining of the downloading request may be implemented by a load balancer, and then:
The load balancer polls each server in the server cluster, determines the server that handles the download request, and causes the server to perform: determining a file identification of the target file according to the downloading request; reading storage event information corresponding to the target file from the database according to the file identification; and extracting the target file from the storage medium based on the storage event information, and feeding back an extraction result to the service system which sends the downloading request.
In one implementation, based on the storage event information, extracting the target file from the storage medium, and feeding back an extraction result to the service system that issues the download request, where the method includes:
extracting a target file from a storage medium based on the storage event information;
if the target file has the preset mark, feeding back information of failure in extracting the target file to a service system which sends a downloading request.
The preset mark may be to mark that the target file is not generated or that the generated target file has an error. In the implementation manner, the information of failure in extracting the target file is fed back to the service system sending the downloading request, so that the user is prevented from downloading the file with the preset mark.
Example 3
In order to solve the problem of low file processing efficiency in the prior art, this embodiment 3 proposes a file processing apparatus, in view of the same inventive concept as the previous embodiment.
As shown in fig. 10, the processing apparatus includes specific components including:
an information acquisition unit 101 that acquires storage event information sent by each service system; and storing the event information, namely storing the files generated in the respective operation process to the corresponding storage medium for each service system, and sending information for indicating the file identification and the file storage path.
The information acquisition unit 101 is further configured to acquire storage event information through a message queue cluster.
An information processing unit 102 writing the stored event information into a database; and the database is opened to users for access so that the users can acquire files generated in the running process of each service system based on the information.
The information processing unit 102 further receives the stored event information through a load balancer, and the load balancer determines a server for processing the stored event information according to the idle state of each server in the server cluster, so that the server writes the stored event information into the database.
In order to realize the processing of the file downloading request, the file processing device of the embodiment further comprises a request acquisition unit, a request analysis unit, a database access unit and a file extraction unit.
The request acquisition unit is used for acquiring a downloading request of the target file; the downloading request is sent out through the service system; the request analysis unit determines the file identification of the target file according to the downloading request; reading storage event information corresponding to the target file from the database according to the file identification; and extracting the target file from the storage medium based on the storage event information, and feeding back an extraction result to the service system which sends the downloading request.
Embodiments of the present application also provide a computing device, in view of the same inventive concepts as the previous embodiments of the present application.
As shown in fig. 11, the computing device includes: a memory 111 and a processor 112. The memory 111 may be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on an electronic device. The memory 111 may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
A processor 112 coupled to the memory 111 for executing a computer program stored in the memory 111 for performing a file processing method as described in the previous embodiments.
The processor 112 may perform other functions in addition to the above functions when executing the computer program in the memory 111, and in particular, reference is made to the description of the foregoing embodiments.
Further, as shown in fig. 11, the computing device further includes: a display 114, a communication component 113, a power supply component 115, an audio component 116, and other components. Only some of the components are schematically shown in fig. 11, which does not mean that the computing device only includes the components shown in fig. 11.
Accordingly, the embodiments of the present application also provide a computer-readable storage medium storing a computer program, which when executed by a computer is capable of implementing the method provided in each of the above embodiments.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.