Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The Json Schema is a Json mode, and refers to a self-defined or appointed Json data format, including members, structures, types, constraints and the like, and can be used for verifying the Json data format, wherein the Json Schema in the invention is a Json data format appointed by a rear end and a front end. Because the JSON Schema is a JSON format itself and is only used for describing another JSON data, the JSON Schema can be used for executing code verification and can also be used as a code document for front-end and back-end development.
Fig. 1 is a schematic diagram of main flow of a data processing method according to an embodiment of the present invention, and as shown in fig. 1, the data processing method of the present invention includes:
step S101, receiving data to be processed returned by the back end.
In the embodiment of the invention, the data to be processed returned by the rear end is received through the intermediate server, the data to be processed is processed, and the processed data is returned to the front end.
Step S102, determining first data and/or second data from the data to be processed according to a preset data structure corresponding to the front end, wherein the first data are data matched with the data structure, and the second data are data not matched with the data structure.
In the embodiment of the invention, whether the data to be processed is matched with the data structure is checked according to the data type, the data value and/or the range of the data value and/or the existence attribute corresponding to the data to be processed so as to determine the first data and/or the second data from the data to be processed.
In the embodiment of the invention, the data to be processed can be multi-layer data, and when the data to be processed is verified, the multi-layer data to be processed needs to be verified step by step.
In the embodiment of the invention, the data structure is in a Json data format. The preset data structure corresponding to the front end may be a Json Schema rule pre-agreed by the back end and the front end. Specifically, when the front end and the back end communicate data in the Json data format, rules of the data structure are determined according to the conventions of Json Schema, including data types, value ranges, default values, whether they are necessary, descriptions, titles, and the like.
In the embodiment of the present invention, as shown in fig. 2, the data format specifying method of the present invention includes the following steps:
in step S201, the backend creates an API interface corresponding to the front end.
In the embodiment of the invention, in order to transmit data according to the agreed data format, the back end is responsible for creating an API interface corresponding to one or more front ends for transmitting data to the front ends.
In step S202, the back end or the front end composes JSON Schema rules.
In the embodiment of the invention, a developer of the back end or the front end writes a JSON Schema rule, so that the back end and the front end transmit data according to a contracted JSON data format.
In the embodiment of the invention, the JSON Schema rule may be:
accordingly, the required object data is:
The data type of "name" is "string", i.e. character string ", the data type of" price "is" number ", i.e. number", the data type of "child" is "array", i.e. array ", the range of data values of" price "includes that the minimum value is 0, and" name "," price "and" child "all belong to necessary fields corresponding to the existence attribute of the data format.
In step S203, the JSON data format is determined.
In the embodiment of the invention, the back end or the front end determines the JSON data format according to the rule of writing the JSON Schema.
Step S204, outputting the JSON Schema.
In the embodiment of the invention, the rear end or the front end responsible for writing the JSON Schema rule outputs the JSON Schema, for example, the JSON Schema is output in the form of an interface or a document, so that a developer at the rear end or the front end can update according to a specific use scene or correct abnormal data and the like.
Further, if the API interface needs to be adjusted, the JSON Schema rule is rewritten.
According to the embodiment of the invention, through the data format appointing method, the data format interacted by the front end and the back end can be clearly described, and the expected data interacted by the front end and the back end is definitely accordant with the rule of the agreed JSON Schema, so that the back end outputs data according to the agreed data format, and the front end receives the data accordant with the agreed data formats of the two parties.
In the embodiment of the present invention, for example, the to-be-processed data received by the intermediate server and returned by the back end is:
The method comprises the steps of checking David according to a data type 'string' corresponding to 'name', determining that the data type of David is 'string', matching a data structure, determining that 'name' is David is first data, checking null according to a data type 'number' corresponding to 'price', determining that the data type of null is not 'number', not matching the data structure, determining that 'price' is second data, taking 'child' as an essential field, and determining that the existence attribute of the 'child' is abnormal when the data to be processed is not in the data corresponding to the 'child' field.
And step S103, converting the second data into target data matched with the data structure under the condition that the second data exists in the data to be processed.
In the embodiment of the invention, the second data is data which is not matched with the data structure, and the intermediate server converts the second data into target data matched with the data structure under the condition that the second data exists in the data to be processed. However, in the process of converting the second data, there may be situations that the second data cannot be converted, or that abnormal data which does not conform to the preset data structure exists after the conversion, and therefore, the data content corresponding to the abnormal data needs to be reported to the back end, so that a developer at the back end can correct the abnormal data, and the corrected data content conforms to the preset data structure.
In the embodiment of the present invention, as shown in fig. 3, the method for reporting abnormal data of the present invention includes the following steps:
Step S301, determining whether the data content of the second data is abnormal, if so, going to step S302, and if not, going to step S310.
In the embodiment of the invention, the intermediate server automatically checks whether the data content of the second data accords with the expected value. For example, the data to be processed received by the intermediate server and returned by the back end is:
Wherein, the data content null corresponding to the "price" is abnormal, and the data corresponding to the "child" field is absent, and the data content of the "child" is abnormal.
Step S302, determining abnormal data.
Step S303, judging whether the continuous reporting times of the abnormal data are larger than a preset time threshold, if so, turning to step S304, and if not, turning to step S305.
Step S304, when the continuous reporting times of the abnormal data are larger than a preset times threshold, taking preset reference data as the target data, so that the front end renders a prompt page according to the reference data.
In the embodiment of the invention, the front end performs fault tolerance processing on the abnormal data, wherein the fault tolerance processing refers to displaying the abnormal data through a spam scheme, for example, the front end renders an object name obj. Name according to the empty string, the received abnormal data is "undefined", if the front end renders according to the "undefined", the front end may possibly generate an abnormality, so that the user experience is poor, the front end may process the "undefined" into { }, and then process the obj. Name into "", i.e. an empty string, i.e. a corresponding spam scheme, so that the front end renders the page according to the empty string, thereby preventing occurrence of an abnormal error phenomenon, or the front end may render the abnormal data into an error prompt page (for example, "error", or x picture, etc.).
In the embodiment of the invention, reference data can be preset, the abnormal data is corrected to be the reference data under the condition that the continuous reporting frequency of the abnormal data is larger than the preset frequency threshold value, the reference data is used as target data to be sent to the front end, and the front end renders a prompt page according to the reference data, wherein the reference data is spam data, such as'.
In step S305, the abnormal data is reported to the back end, so that the back end corrects the abnormal data.
In the embodiment of the invention, the abnormal data can be a hierarchical structure of the data to be processed, a data field, a numerical value corresponding to the data field, an interface name, an IP corresponding to the front end, browser information or operating system information, and the like. The abnormal data can be sent to the back end in a message mode, so that the back end analyzes the message and determines the abnormal data. The reporting service, parameter format and the like of the abnormal data are not limited, and the analysis mode of the data is determined according to the rear end of the reporting system.
For example, the backend corrects the data content corresponding to "price" to "10", and supplements the "child" field and its corresponding data content.
In the embodiment of the invention, the abnormal data can be reported by calling a reporting callback function.
Step S306, the number of times of reporting the abnormal data is increased.
Step S307, the corrected abnormal data sent by the back end is received.
In the embodiment of the invention, when the data content of the corrected abnormal data is normal, the corrected abnormal data is taken as target data, and the target data is sent to the front end.
Step S308, determining whether the data content of the corrected abnormal data is abnormal, if so, turning to step S302, and if not, turning to step S309.
Step S309, converting the corrected abnormal data into target data matched with the data structure.
In the embodiment of the invention, the target data is sent to the front end, so that the front end renders the page according to corrected abnormal data matched with the data structure.
In step S310, the second data is converted into target data matched with the data structure.
In the embodiment of the invention, under the condition that the data content of the second data is normal, the second data is taken as the target data, and the target data is sent to the front end, so that the front end renders the page according to the second data matched with the data structure.
According to the method for reporting the abnormal data, the back end can determine the abnormal data according to the reporting information and help developers to quickly locate data problems including but not limited to API addresses, request parameters, front end information and the like, on the other hand, when the abnormal data is reported, the page of the front end is not misplaced, so that a user is not felt about the abnormal data, user experience is improved, on the other hand, the front end only needs to conduct rendering, development and the like according to the data conforming to the preset data structure and does not need to conduct data structure judgment, so that the developers of the front end do not need to pay excessive attention to the data structure, the development of business logic can be focused, the development efficiency is improved, and the abnormal error reporting condition of a program is reduced.
In the embodiment of the invention, the intermediate server can pre-configure the reporting service (such as the reporting callback function) and the agreed data structure, and package the reporting service and the agreed data structure into a component.
Further, the intermediate server may select to turn on or off the reporting service, select to report or not report the abnormal data, and under the condition that the abnormal data is not reported, may send the abnormal data to the front end as target data, so that the front end renders the prompt page according to the abnormal data or corrects the abnormal data as reference data, and send the reference data to the front end as target data, so that the front end renders the page according to the reference data.
In the embodiment of the invention, the intermediate server can convert the second data by introducing the conversion module, and report the abnormal data which does not accord with the expected value. For example, the conversion module is data=overt (data, schema), converts the data according to the Json schema, and reports the abnormal data which does not accord with the expected value in the data.
Step S104, the first data and/or the target data are sent to the front end, so that the front end renders the page according to the first data and/or the target data.
In the embodiment of the invention, the first data and/or the second data are determined from the data to be processed according to the preset data structure corresponding to the front end by receiving the data to be processed returned by the back end, wherein the first data are data matched with the data structure, the second data are data which are not matched with the data structure, under the condition that the second data exist in the data to be processed, the first data and/or the target data are transmitted to the front end, so that the front end performs steps of rendering pages according to the first data and/or the target data, and the like, the data can be converted into the preset data structure before the front end acquires the data, the data structure processing logic is extracted from the front end, the front end can concentrate on self service development, the fault tolerance processing of mass data is not required, the development cost of the front end is reduced, frequent report phenomenon is prevented, the front end experience is improved, and the front end stability is improved.
Fig. 4 is a schematic diagram of main modules of a data processing apparatus according to an embodiment of the present invention, and as shown in fig. 4, a data processing apparatus 400 of the present invention includes:
And the receiving module 401 is used for receiving the data to be processed returned by the back end.
In the embodiment of the present invention, the receiving module 401 of the intermediate server receives the data to be processed returned by the back end, processes the data to be processed, and returns the processed data to the front end.
The judging module 402 is configured to determine, according to a preset data structure corresponding to the front end, first data and/or second data from the data to be processed, where the first data is data matched with the data structure, and the second data is data not matched with the data structure.
In this embodiment of the present invention, the determining module 402 checks whether the data to be processed matches the data structure according to the data type, the data value, and/or the range of the data value, and/or the presence attribute corresponding to the data to be processed, so as to determine the first data and/or the second data from the data to be processed.
In the embodiment of the invention, the data to be processed can be multi-layer data, and when the data to be processed is verified, the multi-layer data to be processed needs to be verified step by step.
In the embodiment of the invention, the data structure is in a Json data format. The preset data structure corresponding to the front end may be a Json Schema rule pre-agreed by the back end and the front end. Specifically, when the front end and the back end communicate data in the Json data format, rules of the data structure are determined according to the conventions of Json Schema, including data types, value ranges, default values, whether they are necessary, descriptions, titles, and the like.
And the data processing module 403 is configured to convert the second data into target data matched with the data structure when the second data exists in the data to be processed.
In the embodiment of the present invention, the second data is data that does not match the data structure, and the data processing module 403 of the intermediate server converts the second data into the target data that matches the data structure when determining that the second data exists in the data to be processed. However, in the process of converting the second data, there may be situations that the second data cannot be converted, or that abnormal data which does not conform to the preset data structure exists after the conversion, and therefore, the data content corresponding to the abnormal data needs to be reported to the back end, so that a developer at the back end can correct the abnormal data, and the corrected data content conforms to the preset data structure.
And the sending module 404 is configured to send the first data and/or the target data to the front end, so that the front end renders a page according to the first data and/or the target data.
In the embodiment of the invention, the modules such as the receiving module, the judging module, the data processing module and the sending module can convert the data into the preset data structure before the front end acquires the data, and the data structure processing logic is extracted from the front end, so that the front end can concentrate on self service development, does not need to pay attention to data structure abnormality and perform fault tolerance processing on mass data, reduces the development cost of the front end, prevents frequent abnormal error reporting phenomenon, improves user experience and improves the stability of the front end.
FIG. 5 illustrates an exemplary system architecture suitable for application to a data processing method or data processing apparatus of an embodiment of the present invention, as illustrated in FIG. 5, the exemplary system architecture of a data processing method or data processing apparatus of an embodiment of the present invention includes:
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, an intermediate server 505, and a server 506. The network 504 is a medium used to provide communication links between the terminal devices 501, 502, 503 and the intermediate server 505, and between the intermediate server 505 and the server 506. The network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with a server 506 through a network 504 using terminal devices 501, 502, 503 to receive or send messages, etc. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 501, 502, 503.
The terminal devices 501, 502, 503 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 506 may be a server providing various services, such as a background management server providing support for web browser-like websites browsed by the user using the terminal devices 501, 502, 503. The background management server may analyze the received data query request and the like, and feed back the processing result (for example, data to be processed) to the intermediate server 505.
The intermediate server 505 may be a server that provides data processing, such as a background management server that provides support for data format conversion of the data to be processed returned by the server 506. The background management server may perform analysis and other processing on the received data format conversion request and the like, and feed back the processing result (e.g., target data conforming to a preset data format) to the terminal devices 501, 502, 503.
It should be noted that, the data processing method provided by the embodiment of the present invention is generally executed by the intermediate server 505, and accordingly, the data processing apparatus is generally disposed in the intermediate server 505.
It should be understood that the number of terminal devices, networks, intermediate servers, and servers in fig. 5 are merely illustrative. There may be any number of terminal devices, networks, intermediate servers, and servers, as desired for implementation.
Fig. 6 is a schematic structural diagram of a computer system suitable for a terminal device or a server for implementing an embodiment of the present invention, and as shown in fig. 6, a computer system 600 of a terminal device or a server of an embodiment of the present invention includes:
A Central Processing Unit (CPU) 601, which can execute various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Connected to the I/O interface 605 are an input section 606 including a keyboard, a mouse, and the like, an output section 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like, a storage section 608 including a hard disk, and the like, and a communication section 609 including a network interface card such as a LAN card, a modem, and the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 601.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, a processor may be described as comprising a receiving module, a determining module, a data processing module and a transmitting module. The names of these modules do not in any way constitute a limitation of the module itself, for example, a data processing module may also be described as "a module that converts second data into target data that matches a data structure".
As a further aspect, the invention also provides a computer readable medium which may be comprised in the device described in the above embodiments or may be present alone without being fitted into the device. The computer readable medium carries one or more programs, which when executed by the device, cause the device to include receiving data to be processed returned by a back end, determining first data and/or second data from the data to be processed according to a preset data structure corresponding to a front end, wherein the first data is data matched with the data structure, the second data is data not matched with the data structure, converting the second data into target data matched with the data structure when the second data exists in the data to be processed, and transmitting the first data and/or the target data to the front end, so that the front end renders pages according to the first data and/or the target data.
The existing front end generally adopts a data fault tolerance mode to process object data, and when the data level of the object data returned by a certain interface in the front-back end interaction process is too deep (for example, more than three levels), the front end needs to continuously perform fault tolerance processing on the data. However, as the frequency of updating iteration of the front-end technology is continuously increased, the page interaction of the front-end and the back-end is more complex, the data complexity is higher and higher, the front-end processes the page rendering data, and the fault-tolerant processing is carried out on the data, so that the fault-tolerant processing is required to be carried out on the data obtained or modified each time, the development cost of the front-end is higher and higher, the control on the data is easily lost, and unpredictable errors occur.
According to the technical scheme provided by the embodiment of the invention, through the data structure written according to the JSON Schema, the data to be processed returned by the rear-end interface is automatically converted into the target data conforming to the expected data structure, the front end does not need to continuously perform fault-tolerant processing in the process of acquiring the data or modifying the data, the development cost of the front end is reduced, and meanwhile, the abnormal data not conforming to the expected value is reported, so that the problem data can be rapidly positioned and corrected, the user is not felt to the abnormal data, and the user experience is greatly improved.
According to the technical scheme of the embodiment of the invention, before the front end acquires the data, the data is converted into the preset data structure, and the data structure processing logic is extracted from the front end, so that the front end can concentrate on self service development, does not need to pay attention to data structure abnormality and perform fault tolerance processing on mass data, the development cost of the front end is reduced, frequent abnormality error reporting phenomenon is prevented, user experience is improved, and the stability of the front end is improved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.