Web Map Services implementation method based on the cloud frameworkTechnical field
The present invention relates to Web Map Services field, particularly a kind of Web Map Services implementation method based on the cloud framework.
Background technology
Web Map Services (Web map service is referred to as WMS) is followed the WMS1.1.1 standard of OGC, utilizes the data creating map with geospatial location information.Wherein map is defined as the visual performance of geodata.This normalized definition three operations: GetCapabitities return service level metadata, it is a kind of description to information on services Contents and request parameter; GetMap returns to a map image, and its geographical space reference and size parameter are clearly defined; GetFeatureInfo(is optional) return to the information of some the special key element that is presented on map.For the Web Map Services can be provided, the data volume of storing in the WMS server is both not huge, for example, for NASA LandSat I3 data set, the data volume of the 8th layer of CHINESE REGION (from longitude and latitude (70,5) to longitude and latitude (135,60)) just reaches 100GB, and for each layer data, its volume is 4 times of left and right of last layer.In prior art, take single node as the main framework that carries out, when facing huge memory data output, existing WMS service is difficult to satisfy the strict demand of big data quantity storage and high reliability to the WMS server substantially.
Summary of the invention
The present invention is directed to prior art and have the not high shortcoming of data bottleneck, reliability take single node as main WMS service, provide a kind of based on the cloud concept, can realize the cluster mode of operation of multimachine multinode, have the novel Web Map Services implementation method based on the cloud framework of higher data storage capacities, system load ability and reliability.
For achieving the above object, the present invention can take following technical proposals:
Web Map Services implementation method based on the cloud framework, comprise load equalizer, working node, described working node comprises WMS service module, distributed file system, and described distributed file system comprises distributed file system interface, memory node, and concrete steps are as follows:
Step 1: described load equalizer receives the request of Web Map Services;
Step 2: described load equalizer selects a working node as the working node that receives request according to load balancing, and described request is sent to selected working node;
Step 3: the WMS service module of selected working node is resolved the URL of described request, according to longitude and latitude scope and the resolution of resolving resulting URL parameter positioning map, calculates the needed Figure List of cutting according to described longitude and latitude scope and resolution;
Step 4: described WMS service module is sent to described distributed file system with the described title of cutting figure in Figure List of cutting, described distributed file system cuts accordingly figure according to the name lookup of the described figure of cutting and the figure that cuts that will find returns to described WMS service module, described WMS service module is by the described distributed file system of described distributed file system interface accessing, the described figure of cutting stores with document form, and described distributed file system is by the described file of described storage node accesses;
Step 5: described WMS service module splices the figure that cuts that finds, adjust the resolution of cutting figure after splicing, according to described longitude and latitude scope, the figure that cuts that adjusts after resolution is carried out cutting and obtains the remote sensing image raster data, and described remote sensing image raster data is sent to described load equalizer;
Step 6: described load equalizer returns to the remote sensing image raster data of asking.
As preferably, described load balancing comprises poll, IP-HASH.
As preferably, described working node forms the P2P network.
As preferably, also comprise the double-layer high-speed cache policy, described double-layer high-speed cache policy uses the double-layer high-speed buffer memory, described double-layer high-speed buffer memory comprises the first buffering area that is positioned at described load equalizer and the second buffering area that is positioned at described working node, and described double-layer high-speed cache policy comprises following concrete steps:
In described step 1, described load equalizer detects described remote sensing image raster data in the first buffering area, if described the first buffering area has comprised described remote sensing image raster data execution in step six;
In described step 3, described WMS service module judges one by one to described image list, if described the second buffering area has comprised all figure that cuts, execution in step five; If described the second buffering area not yet comprises all figure that cuts, the title of cutting figure that will not yet be loaded on the second buffering area is sent to described distributed file system;
In described step 5, described load equalizer puts down in writing the remote sensing image raster data that obtains to described the first buffering area.
As preferably, the file that is cached in described double-layer high-speed buffer memory comprises configuration file and remote sensing image raster data.
Load balancing is to improve the key point of whole system computing capability.WMS is that the http agreement by standard communicates.Request for the http agreement is easily to go to realize load balancing.Simultaneously, for applied environment involved in the present invention, each request is independently, is also each request and session irrelevant (session is irrelevant).Such as, the Map Services of request is all relatively independent each time in client, is that milli is unallied each other, and this request can not have influence on other request, and other request can not affect this request yet.Thereby, during the selection of load balancing in the present invention, need not to consider the session problem, and only need consider load and balanced problem.In the present invention, adopted the strategy of poll.The poll strategy is realized simple, and efficient is very high, and the resource that consumes simultaneously is very little, is a kind of strategy of very lightweight, is fit to very much application scenarios of the present invention.
In the present invention, distributed file system is to have adopted the P2P framework, and unconventional client/server.The P2P framework has very large advantage at aspects such as the stability of a system, validity than client/server.In the present invention, adopt the distributed file system of P2P framework to also have a reason to be: in the designed cluster of the present invention, each working node has Three role: the memory node of WMS, distributed file system interface, distributed document.Wherein the memory node of distributed file system interface, distributed document is relevant with file system.If the employing host-guest architecture, in system, also just all leading to host node is used as the distributed file system interface so.Like this, host node is exactly the Single Point of Faliure that can become whole cluster, can make Systems balanth and availability greatly reduce.In the present invention, WMS in each node uses the distributed file system access interface on this node, the machine once the function of the distributed file system node of this node is delayed also cannot use with regard to the WMS on this node, and can not cause all the delay gross error of machine of whole cluster.
In the distributed file system of P2P framework, each node can be the file system access interface, it can be the memory node of distributed file system, and all nodes coact, and jointly safeguard the name space (safeguarding that namely the distributed file system File is to the mapping between the chunk data block and the mapping between chunk data block and memory node) of whole file system.Like this, the status of each node is reciprocity, is all equality, there is no master slave relation.
In WMS used, data only have to be increased and two kinds of situations of deletion, and in the situation that general, the situation that does not basically have the deletion data occurs.Thereby, under these circumstances, be fit to very much use the Cache technology.Cache has very important effect for the overall performance of system.In the cluster of design of the present invention, adopted the strategy of double-deck Cache, i.e. the Cache of the Cache of load equalizer one deck and WMS one deck.
Cache for load equalizer one deck, existed corresponding file (may be configuration file in the Cache of load equalizer discovery at oneself, also may be the remote sensing image of rasterizing), that load equalizer will automatically return to the user with file, and does not carry out load balancing; If there is no corresponding file, load equalizer can remove to carry out load balancing, can be cached returning results.Because load equalizer is unique for whole trooping, thereby its Cache storage capacity is limited.
For the Cache of WMS one deck and, if the WMS server is found to have stored required file in the Cache of this node, the WMS server will directly be got file from Cache so; If file does not exist in Cache, the WMS server can go for and ask distributed file system, and with the file cache of being asked in Cache.Cache in the present invention is the object Cache of pure internal memory operation, with guarantee system than the low-response time.
The present invention has significant technique effect owing to having adopted above technical scheme:
Compared with prior art, the present invention has realized the cluster mode of operation of multimachine multinode, adopt the P2P framework between each working node, when Single Point of Faliure appears in certain working node in cluster, can not affect the performance of other working nodes, avoided existing in traditional host-guest architecture framework because the host node fault causes the delay phenomenon of machine of whole cluster, improved Systems balanth, reliability.
Secondly, adopt the double-layer high-speed cache policy based on the double-layer high-speed buffer memory, shortened the response time of system.
Description of drawings
Fig. 1 is the system architecture schematic diagram of the Web Map Services implementation method based on the cloud framework of the present invention.
Fig. 2 is the schematic flow sheet of embodiment 1.
Embodiment
The present invention is described in further detail below in conjunction with embodiment.
Embodiment 1
Web Map Services implementation method based on the cloud framework comprises load equalizer 1, working node 2, and as shown in Figure 1, described working node 2 forms the P2P network.Wherein, working node 2 comprises WMS service module 21, distributed file system 22, and described distributed file system 22 comprises distributed file system interface 221, memory node 222.
Working node 2 is responsible for processing practically the request that the user submits to.Working node 2 is processed simultaneously and is born: the memory node 222 of WMS, distributed file system interface 221, distributed document.WMS(is Web Map Services Web Map Service) follow the WMS1.1.1 standard of OGC, the data creating map that this service utilization has geospatial location information.Wherein map is defined as the visual performance of geodata.Distribute file system interface 221 (DFS Portal) is the interface of WMS access distributed file system 22, and namely WMS accesses distributed documents by distributed file system interface 221, thereby obtains the remote sensing image of configuration file and rasterizing.The memory node 222 of distributed file system 22 is to be responsible for the node of storage file in the distributed file system cluster.Each WMS uses distributed file system 22 access interfaces of this node, also namely by himself visiting distributed file system 22.Each node has independently function like this, and is also the relation of P2P between other node, for the reliability and availability that improves system, very large help is arranged like this.
Concrete steps are as follows, as shown in Figure 2:
Step 1: the user submits the GetMap request to described load equalizer 1, and load equalizer 1 receives the GetMap request; Described load equalizer 1 detects described remote sensing image raster data in the first buffering area 3, if described the first buffering area 3 has comprised described remote sensing image raster data execution in step six.
step 2: if described the first buffering area 3 not yet comprises described remote sensing image raster data, described load equalizer 1 is according to poll, IP-HASH even load balance policy selects a working node 2 as the working node that receives request, load equalizer 1 checks the operating state of selected working node 2, if this node is working properly, it is the inspection of normal response load equalizer 1, described request is sent to selected working node 2, if the not inspection of normal response load equalizer 1 of working node 2 that should select, repeatedly carry out load balancing and select working node 2.
Step 3: selected working node 2 is accepted the GetMap request, the URL of 21 pairs of described requests of WMS service module resolves, according to longitude and latitude scope and the resolution of resolving resulting URL parameter positioning map, calculate the needed Figure List of cutting according to described longitude and latitude scope and resolution.21 pairs of described image list of described WMS service module judge one by one, cut all figure that cut in Figure List if described the second buffering area has obtained and comprised, and execution in step five; If described the second buffering area not yet comprises all figure that cuts, the title of cutting figure that will not yet be loaded on the second buffering area is sent to described distributed file system 22.
Step 4: cut figure in order to obtain, described WMS service module 21 is sent to described distributed file system 22 with the described title of cutting figure in Figure List of cutting, described distributed file system 22 cuts accordingly figure according to the name lookup of the described figure of cutting and the figure that cuts that will find returns to described WMS service module 21, described WMS service module 21 is by the described distributed file system interface 221 described distributed file systems 22 of access, the described figure of cutting stores with document form, and described distributed file system 22 is by the described memory node 222 described files of access.
Step 5: the figure that cuts that 21 pairs of described WMS service modules find splices, adjust the resolution of cutting figure after splicing, according to described longitude and latitude scope, the figure that cuts that adjusts after resolution is carried out cutting and obtains the remote sensing image raster data, and described remote sensing image raster data is sent to described load equalizer 1, described load equalizer 1 puts down in writing the remote sensing image raster data that obtains to described the first buffering area.
Step 6: described load equalizer 1 returns to the remote sensing image raster data of asking.
In order to improve buffer efficiency, the file that is cached in described double-layer high-speed buffer memory comprises configuration file and remote sensing image raster data.
The contrast experiment 1
Before the technical scheme that the present invention is put down in writing is tested, need to build a suitable cluster environment and test.On hardware, load equalizer 1 in cluster environment and working node 2 all use following hardware environment to build: core frequency is internal memory, the hard disk of 50GB+7200RPM, the gigabit ethernet card of monokaryon CPU, the 1024MB of 2.3GHz, wherein the software environment of working node 2 is as follows: Windows 2008 Server+SUN JDK 1.6, the software environment of load equalizer 1 is as follows: Ubuntu 10.04+SUN JDK 1.6+Nginx 1.2.
Here, we carry out single node and cluster to the response contrast test of single request, the test purpose of this test is in the situation that system's free time, the single node environment that test includes only a working node is to response time of single request and the above-mentioned cluster environment difference for the response time of single request, consideration is under cluster environment, single request is still completed on a working node 2, can expect that the two test result is should difference little.
Test process is as follows: respectively stand-alone environment and cluster environment are sent 6 requests, all ask resolution is 256*256 at every turn, and the longitude and latitude scope is 10 map datum, and test result is described in table 1 below:
Table 1 single node and the cluster response contrast test result to single request
From upper table 1 test result, for single request, the response time of cluster environment and single node environment is roughly the same, this also is consistent with expected result, although the forwarding of load equalizer 1 has been passed through in the request that also illustrates simultaneously, obvious decline does not appear in overall performance, in a word, for single request, the handling property difference of cluster environment and single node environment is also not obvious.
The contrast experiment 2
The cluster scale of the cluster environment response time to concurrent request is compared, and the cluster environment of using is with contrast experiment 1.The test cluster scale is 1,2,3,4 respectively, concurrent quantity is that the concurrent request processing ability of 100 o'clock is recorded respectively the average response time of overall response time, each request and the number of makeing mistakes, here cluster scale represents the number of working node 2 in cluster environment, for example, only has 1 working node 2 in cluster scale 1 expression cluster environment, the cluster environment of this moment just is equivalent to the single node environment, in cluster scale 4 expression cluster environment, 4 parallel working nodes 2 is arranged.Identical in the request that sends to cluster environment and contrast experiment 1, test result is as shown in the table:
Table 2 overall response time
Table 3 average response time
Table 4 number of makeing mistakes
From above result, we can find out, are 100 o'clock at concurrency, and along with the increase of cluster scale, response total time and average response time are shorter and shorter, and the number of makeing mistakes is also fewer and feweri.Although, at cluster scale be 1 o'clock response total time than cluster scale be 2 o'clock little, make mistakes owing to having occurred 63 in 100 requests, average response time has but been grown a lot.From above test result, we can find out, along with the increase of test environment cluster scale, the disposal ability of concurrent request have also been increased.
The contrast experiment 3
This contrast experiment compares test to single node environment and cluster environment to response time of concurrent request, the cluster environment of using is with contrast experiment 1, measurement scope is that 4 cluster and unit are the response condition of 10,20,30,40,50,60 o'clock at concurrency, record respectively overall response time, the average response time of each request and error situation.Each time the map datum of request be distinguished as 256*256, the longitude and latitude scope is 10, namely maximum poor with minimum longitude and latitude is 10.
Table 5 overall response time
Table 6 average response time
Table 7 number of makeing mistakes
As seen from the above table, along with the rising of concurrency, the difference of the concurrent request processing ability of cluster and unit is more and more obvious.And we can see that the cluster of 4 nodes is than very significantly advantage is arranged on unit on performance.Simultaneously, unit is 60 o'clock at concurrency, occured to make mistakes, and cluster is still very sane, mistake substantially do not occur.
In a word, the above is only preferred embodiment of the present invention, and all equalizations of doing according to the present patent application the scope of the claims change and modify, and all should belong to the covering scope of patent of the present invention.