Movatterモバイル変換


[0]ホーム

URL:


CN107357794A - Optimize the method and apparatus of the data store organisation of key value database - Google Patents

Optimize the method and apparatus of the data store organisation of key value database
Download PDF

Info

Publication number
CN107357794A
CN107357794ACN201610305828.7ACN201610305828ACN107357794ACN 107357794 ACN107357794 ACN 107357794ACN 201610305828 ACN201610305828 ACN 201610305828ACN 107357794 ACN107357794 ACN 107357794A
Authority
CN
China
Prior art keywords
prefix
keyword
list
database
hit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610305828.7A
Other languages
Chinese (zh)
Other versions
CN107357794B (en
Inventor
黄肖明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co LtdfiledCriticalBeijing Jingdong Century Trading Co Ltd
Priority to CN201610305828.7ApriorityCriticalpatent/CN107357794B/en
Publication of CN107357794ApublicationCriticalpatent/CN107357794A/en
Application grantedgrantedCritical
Publication of CN107357794BpublicationCriticalpatent/CN107357794B/en
Activelegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Classifications

Landscapes

Abstract

This application discloses the method and apparatus of the data store organisation of optimization key value database.One embodiment of methods described includes:The keyword of predetermined number is read from database and each keyword to reading performs analytical procedure up to all keywords in analytical database, obtains prefix list;Obtain the quantity and memory space of the keyword of the prefix in hit prefix list;The quantity and memory space of the keyword of prefix in hit prefix list, optimize the data store organisation of database.The embodiment improves the storage performance of database and accelerates the reading efficiency of database.

Description

Optimize the method and apparatus of the data store organisation of key value database
Technical field
The application is related to field of computer technology, and in particular to Internet technical field, especially relates toAnd the method and apparatus of the data store organisation of optimization key value database.
Background technology
With the complexity more and more higher of Distributed Storage business, in order to avoid single commonIt is excessive to enjoy reading pressure on internal memory, shared storage can be set by Hash (hash) scheduling algorithmIt is standby to carry out load balancing, but this also brings the sharp increase of key assignments table quantity.
At present, during key assignments table in monitoring distributed data storage, all keys are generally obtainedWord and the memory space and data structure for showing keyword one by one, or for pass that user specifiesKey word obtains the data structure of memory space and keyword, or should by line real-time monitoringWith read-write keyword, and the service condition such as reading performance of analysis keyword in real time.
However, it is only certain data line in key assignments table to monitor data acquired during key assignments table at presentThe information such as the memory space of occupancy or read-write are frequent, can not be deposited according to the data-optimized data of acquisitionStorage.
The content of the invention
The purpose of the application is to propose a kind of data storage knot of improved optimization key value databaseThe method and apparatus of structure, to solve the technical problem that background section above is mentioned.
In a first aspect, this application provides a kind of data store organisation for optimizing key value databaseMethod, methods described include:The keyword of predetermined number is read from database and to readingEach keyword performs analytical procedure up to all keywords in analytical database, describedAnalytical procedure includes:The keyword read is analyzed to be segmented, to the participle perform detectionStep, the detecting step include:Last participle is deleted to obtain predicting prefix, identificationWhether the prediction prefix hits the prefix in prefix list, if hit, the prefix is arrangedThe number that is hit of prefix in table adds 1, if miss, the prediction prefix is added toThe prefix list, and identify whether the prediction prefix is single participle, if so, then to readingThe next keyword taken performs analytical procedure, if it is not, then to the participle in the prediction prefixPerform the detecting step;Obtain the quantity for the keyword for hitting the prefix in the prefix listAnd memory space;The quantity of the keyword of prefix in the hit prefix list andMemory space, optimize the data store organisation of database.
In certain embodiments, it is described to obtain the keyword for hitting the prefix in the prefix listQuantity and memory space include:Before number is hit in the prefix list less than 2 timesKeyword corresponding to sewing re-executes analytical procedure, the prefix list after being updated;Obtain lifeDescribed in update after prefix list in prefix keyword quantity and memory space;AndThe quantity of the keyword of the prefix in the hit prefix list and storage are emptyBetween, optimizing the data store organisation of database includes:Arranged according to the prefix after the hit renewalThe quantity and memory space of the keyword of prefix in table, optimize the data store organisation of database.
In certain embodiments, the keyword of predetermined number is read from database and to readingEach keyword performs analytical procedure until all keywords in analytical database include:The keyword of predetermined number is read from database and analysis is performed to each keyword of readingStep, at the end of analytical procedure is performed to the keyword of predetermined number, by the prefix listMiddle prefix of the hit mark less than 2 times is moved in individual Hash table as individual prefix until having dividedAnalyse all keywords in database;It is described to obtain the pass for hitting the prefix in the prefix listThe quantity and memory space of key word include:After all keywords in identifying the database,After pair key corresponding with the individual prefix re-executes the analytical procedure to be updatedThe quantity of the keyword of the prefix in the prefix list after the renewal is hit in prefix list, acquisitionAnd memory space;And the keyword of the prefix in the hit prefix listQuantity and memory space, optimizing the data store organisation of database includes:According to hitThe quantity and memory space of the keyword of the prefix in prefix list after renewal, optimize databaseData store organisation.
In certain embodiments, the database includes example and/or memory image file.
In certain embodiments, the keyword that the analysis is read is included with obtaining participle:
In certain embodiments, the pass of the prefix in the hit prefix listThe quantity and memory space of key word, the data store organisation of the optimization database include:According toThe quantity and memory space of the keyword of prefix in the hit prefix list, to userThe option of optimization data store organisation is presented;In response to receiving selection of the user to the option,Optimize the data store organisation of the database.
Second aspect, this application provides a kind of data store organisation for optimizing key value databaseDevice, described device include:Analytic unit, for reading the pass of predetermined number from databaseKey word simultaneously performs analytical procedure until the institute in analytical database to each keyword of readingThere is keyword, the analytical procedure includes:The keyword read is analyzed to be segmented, to instituteParticiple perform detection step is stated, the detecting step includes:Last participle is deleted to obtainPrefix is predicted, identifies whether the prediction prefix hits the prefix in prefix list, if hit,The number that is hit of the prefix in the prefix list is then added 1, if miss, by described inPrediction prefix is added to the prefix list, and identifies whether the prediction prefix is single participle,If so, analytical procedure then is performed to next keyword of reading, if it is not, then to the predictionParticiple in prefix performs the detecting step;Acquiring unit, the prefix is hit for obtainingThe quantity and memory space of the keyword of prefix in list;Optimize unit, for according toThe quantity and memory space of the keyword of the prefix in the prefix list are hit, optimizes databaseData store organisation.
In certain embodiments, the acquiring unit is further used for:To in the prefix listKeyword corresponding to being hit prefix of the number less than 2 times re-executes analytical procedure, obtains morePrefix list after new;Obtain the keyword of the prefix in the prefix list after hitting the renewalQuantity and memory space;And the optimization unit is further used for:According to hit moreThe quantity and memory space of the keyword of the prefix in prefix list after new, optimize databaseData store organisation.
In certain embodiments, the analytic unit is further used for:Read from database pre-If the keyword of quantity simultaneously performs analytical procedure to each keyword of reading, to present countAt the end of the keyword of amount performs analytical procedure, hit mark in the prefix list is less than 2Secondary prefix is moved in individual Hash table as individual prefix until all in analytical databaseKeyword;The acquiring unit is further used for:Pair key weight corresponding with the individual prefixThe analytical procedure is newly performed with the prefix list after being updated, is obtained after hitting the renewalPrefix list in prefix keyword quantity and memory space;And the optimization unitIt is further used for:The number of the keyword of prefix in the prefix list after the hit renewalAmount and memory space, optimize the data store organisation of database.
In certain embodiments, in some optional implementations of the present embodiment, the analysisThe keyword that unit is used to read predetermined number from the database includes:The analytic unitFor reading the keyword of predetermined number from example and/or memory image file.
In certain embodiments, the analytic unit is used to analyze the keyword read to be dividedWord includes:The analytic unit be used for according to it is following it is one or more analysis read keywords withSegmented:Separator, capital and small letter change and digital alphabet change.
In certain embodiments, the optimization module is further used for:According to the hitThe quantity and memory space of the keyword of prefix in prefix list, optimization data are presented to userThe option of storage organization;In response to receiving selection of the user to the option, optimize the dataThe data store organisation in storehouse.
The method and apparatus of the data store organisation for the optimization key value database that the application provides, it is firstThe keyword of predetermined number is first read from database and each keyword to reading is performed and dividedStep is analysed until all keywords in analytical database, the analytical procedure include:AnalysisThe keyword of reading is to be segmented, to the participle perform detection step, the detecting stepIncluding:Last participle is deleted to obtain predicting prefix, identifies whether the prediction prefix ordersPrefix in middle prefix list, if hit, by being hit for the prefix in the prefix listNumber adds 1, if miss, the prediction prefix is added into the prefix list, and knowWhether not described prediction prefix is single participle, if so, then being held to next keyword of readingRow analytical procedure, if it is not, then performing the detecting step to the participle in the prediction prefix;The quantity and memory space for the keyword for hitting the prefix in the prefix list are obtained afterwards;MostThe quantity and memory space of the keyword of prefix in the hit prefix list afterwards,Optimize the data store organisation of key value database, so as to improve the storage performance of database and addThe speed reading efficiency of database.
Brief description of the drawings
Retouched by reading with reference to the detailed of being made to non-limiting example of being made of the following drawingsState, other features, objects and advantages will become more apparent upon:
Fig. 1 is that the application can apply to exemplary system architecture figure therein;
Fig. 2 is one according to the method for the data store organisation of the optimization key value database of the applicationThe flow chart of individual embodiment;
Fig. 3 be according to the method for the data store organisation of the optimization key value database of the application againThe flow chart of one embodiment;
Fig. 4 is according to the method for the data store organisation of the optimization key value database of the applicationThe flow chart of three embodiments;
Fig. 5 is one according to the device of the data store organisation of the optimization key value database of the applicationThe structural representation of individual embodiment;
Fig. 6 is adapted for for realizing the terminal device of the embodiment of the present application or the computer of serverThe structural representation of system.
Embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is appreciated that, specific embodiment described herein is used only for explaining related invention, rather than to the hairBright restriction.It also should be noted that for the ease of description, illustrate only in accompanying drawing withAbout the related part of invention.
It should be noted that in the case where not conflicting, embodiment and embodiment in the applicationIn feature can be mutually combined.Describe this in detail below with reference to the accompanying drawings and in conjunction with the embodimentsApplication.
Fig. 1 is shown can be using the data store organisation of the optimization key value database of the applicationThe example system of the embodiment of the device of the data store organisation of method or optimization key value databaseFramework 100.
As shown in figure 1, system architecture 100 can include terminal device 101,102,103,Network 104 and server 105,106.Network 104 is in terminal device 101,102,103The medium of communication link is provided between server 105.Network 104 can include various connectionsType, such as wired, wireless communication link or fiber optic cables etc..
User 110 can pass through network 104 and service with using terminal equipment 101,102,103Device 105,106 interacts, to receive or send message etc..Terminal device 101,102,103On various telecommunication customer end applications can be installed, such as the application of data base administration class, shopping classUsing, searching class application, JICQ, mailbox client, social platform software etc..
Terminal device 101,102,103 can be the various electronic equipments for having display screen, bagInclude but be not limited to smart mobile phone, tablet personal computer, E-book reader, MP3 player (MovingPicture Experts Group Audio Layer III, dynamic image expert's compression standard audio layerFace 3), (Moving Picture Experts Group Audio Layer IV, dynamic image are special by MP4Family's compression standard audio aspect 4) player, pocket computer on knee and desktop computer etc..
Server 105,106 can be to provide the server of various database services, such as to endEnd equipment 101,102,103 provides the background server supported.Background server can dockThe data such as the request received are carried out the processing such as analyzing, and result is fed back into terminal device.
It should be noted that the data for the optimization key value database that the embodiment of the present application is provided are depositedThe method of storage structure is typically performed by terminal device 101,102,103 or server 105,106,Correspondingly, the device for optimizing the data store organisation of key value database is generally positioned at terminal device101st, 102,103 or server 105,106 in.
It should be understood that the number of the terminal device, network and server in Fig. 1 is only to illustrateProperty.According to needs are realized, can have any number of terminal device, network and server.
It should be appreciated that the key value database in the application refers to key-value (Key-Value) dataStorehouse, the Key-Value databases in the prior art and in the technology of future development can be included.Below by taking the Redis databases in Key-Value databases as an example, illustrate to optimize databaseThe method and apparatus of data store organisation.
Fig. 2 is refer to, Fig. 2 is shown to be deposited according to the data of the optimization key value database of the applicationThe flow 200 of one embodiment of the method for storage structure.The data of the optimization key value database are depositedThe flow of the method for storage structure, by taking Redis databases as an example, specifically include following steps:
In step 210, the keyword (Key) of predetermined number is read from Redis databasesAnd analytical procedure is performed until having analyzed in Redis databases to each keyword of readingAll keywords.
In the present embodiment, analytical procedure includes:The keyword read is analyzed to be segmented, it is rightPerform detection step is segmented, detecting step includes:Before last participle is deleted to obtain predictionSew, whether identification prediction prefix hits the prefix in prefix list, if hit, prefix is arrangedThe number that is hit of prefix in table adds 1, if miss, prediction prefix is added into prefixList, and whether identification prediction prefix is single participle, if so, then to next pass of readingKey word performs analytical procedure, if it is not, then to the participle perform detection step in prediction prefix.
When reading the keyword of predetermined number from Redis databases, in order to prevent what is readRedis data volumes are excessive, cause application tool internal memory to exhaust, can every time from Redis examples and/ or Redis memory image files in read predetermined number keyword.
, can be according to following one or more points when the keyword that analysis is read is to obtain participleThe keyword read is analysed to be segmented:Separator, capital and small letter change and digital alphabet change.For example, it can be segmented according to separator analysis keyword Customer-username" Customer " and " username ";Can be according to capital and small letter mutation analysis keywordCustomerSaveKeyUsername, segmented:“Customer”、“Save”、“Key”" Username ";Participle keyword " abc1231 " can be changed according to digital alphabet, obtainedSegment " abc ", " 123 " and " 1 ".Herein, it is also possible to which the keyword of reading is the overall situationUser identity proves (UID) class keyword, then global user identity can be proved into class is crucialWord is as a kind of keyword individually recorded.
It should be appreciated that above-mentioned detecting step prefix list when performing first time is sky, now in advanceThe prefix surveyed in the miss prefix list of prefix, prediction prefix is added to prefix list afterwards,Whether then follow-up participle can hit prefix in perform detection step with identification prediction prefixPrefix in list.
In a step 220, obtain the quantity of the keyword of the prefix in hit prefix list and depositStore up space.
In the present embodiment, based on all keys identified in step 210 in Redis databasesThe prefix list that word obtains, Key all in Redis is traveled through, obtained in hit prefix listPrefix Key and Key value (Value), and record each Key and Value and takeMemory headroom.
, can in the quantity and memory space of the keyword of the prefix in obtaining hit prefix listTo obtain the quantity and memory space of the keyword for hitting all prefixes in prefix list, so as toImprove the comprehensive of the data of acquisition;The number of the prefix in hit prefix list can also be obtainedMore than the quantity and memory space of the keyword of pre-determined number (such as 2 times), before hitSew quantity of the number not less than the keyword of pre-determined number (such as 2 times) of the prefix in listAnd memory space, this partial data individually can be obtained or abandon, so as to improve the effect of acquisitionRate.
In step 230, according to hit prefix list in prefix keyword quantity and depositSpace is stored up, optimizes the data store organisation of Redis databases.
In the present embodiment, based on the prefix in the hit prefix list obtained in a step 220Keyword quantity and memory space, optimize Redis databases data store organisation.ShowExample property, can according to the quantity of the keyword of the prefix in the hit prefix list of acquisition with depositSpace is stored up, the load pressure of caching system is prompted, to carry out load balancing;Can also basisThe quantity and memory space of the keyword of prefix in the hit prefix list of acquisition, optimize dataStorage, such as delete junk data or remove duplicate data etc..
In the quantity and memory space of the keyword of the prefix in hit prefix list, optimization, can be in the key of the prefix in hitting prefix list during the data store organisation of Redis databasesWhen the quantity and memory space of word meet the preparatory condition of preset rules, complete to limit in preset rulesFixed operation, can also so as to be optimized to the data store organisation in Redis databasesThe quantity and memory space of the keyword of prefix in above-mentioned hit prefix list meet defaultDuring condition, the option of optimization data store organisation is presented to user, in response to receiving user to choosingThe selection of item, optimize the data store organisation of Redis databases.
The above embodiments of the present application optimize the data store organisation of Redis databases, improveThe storage performance of Redis databases and the reading efficiency for accelerating Redis databases.
With further reference to Fig. 3, it illustrates the data of the optimization key value database according to the applicationThe flow chart 300 of another embodiment of the method for storage organization.The optimization key value databaseThe flow 300 of the method for data store organisation, still by taking Redis databases as an example, specifically include withLower step:
In the step 310, the keyword of predetermined number is read from Redis databases and to readingEach keyword taken performs analytical procedure until the institute analyzed in Redis databases is relevantKey word.
In the present embodiment, analytical procedure includes:The keyword read is analyzed to be segmented, it is rightPerform detection step is segmented, detecting step includes:Before last participle is deleted to obtain predictionSew, whether identification prediction prefix hits the prefix in prefix list, if hit, prefix is arrangedThe number that is hit of prefix in table adds 1, if miss, prediction prefix is added into prefixList, and whether identification prediction prefix is single participle, if so, then to next pass of readingKey word performs analytical procedure, if it is not, then to the participle perform detection step in prediction prefix.
It should be appreciated that the step 310 in the present embodiment is identical with the step 210 in Fig. 2, becauseThis operation being described in step 210 and feature are equally applicable to step 310, no longer superfluous hereinState.
In step 320, to being hit in prefix list corresponding to prefix of the number less than 2 timesKeyword re-executes analytical procedure, the prefix list after being updated.
In the present embodiment, in order to improve identification Redis databases in all keywords standardExactness, can be to being hit keyword weight corresponding to prefix of the number less than 2 times in prefix listIt is new to perform analytical procedure, with the prefix list after being updated.
In a step 330, the keyword of the prefix in the prefix list after hit renewal is obtainedQuantity and memory space.
In the present embodiment, based on the prefix pair for being less than 2 times in step 320 to being hit numberThe keyword answered re-executes the prefix list after the renewal that analytical procedure obtains, and travels through RedisIn all keyword Key, obtain the Key of prefix in the prefix list after hit renewal withAnd Key value Value, and record the memory headroom that each Key and Value take.
In step 340, the keyword of the prefix in prefix list after being updated according to hitQuantity and memory space, optimize the data store organisation of Redis databases.
In the present embodiment, based on the prefix list after the hit renewal obtained in a step 330In prefix keyword quantity and memory space, the data of Redis databases can be optimizedStorage organization.It is exemplary, can be according to before in the prefix list after the hit renewal of acquisitionThe quantity and memory space for the keyword sewed, the load pressure of caching system is prompted, to carry outLoad balancing;The key of the prefix in prefix list after can also being updated according to the hit of acquisitionThe quantity and memory space of word, optimize data storage, such as delete junk data or remove and repeatData etc..
From figure 3, it can be seen that compared with embodiment corresponding to Fig. 2, it is excellent in the present embodimentThe flow 300 for changing the method for caching Redis data store organisation is highlighted to being hit numberKeyword corresponding to prefix less than 2 times re-executes analytical procedure with the prefix after being updatedThe step of list.Thus, after the scheme of the present embodiment description can improve identification hit renewalThe quantity of the keyword of prefix in prefix list and the precision of memory space, to optimize RedisThe data store organisation of database provides more accurately data.
With further reference to Fig. 4, it illustrates the data of the optimization key value database according to the applicationThe flow chart 400 of 3rd embodiment of the method for storage organization.The optimization key value databaseThe flow 400 of the method for data store organisation, continues by taking Redis databases as an example, including followingStep:
In step 410, the keyword of predetermined number is read from Redis databases and to readingEach keyword taken performs analytical procedure.
In the present embodiment, analytical procedure includes:The keyword read is analyzed to be segmented, it is rightPerform detection step is segmented, detecting step includes:Before last participle is deleted to obtain predictionSew, whether identification prediction prefix hits the prefix in prefix list, if hit, prefix is arrangedThe number that is hit of prefix in table adds 1, if miss, prediction prefix is added into prefixList, and whether identification prediction prefix is single participle, if so, then to next pass of readingKey word performs analytical procedure, if it is not, then to the participle perform detection step in prediction prefix.
When reading the keyword of predetermined number from Redis databases, in order to prevent what is readRedis data volumes are excessive, cause application tool internal memory to exhaust, can every time from Redis examples and/ or Redis memory image files in read predetermined number keyword.
, can be according to following one or more points when the keyword that analysis is read is to obtain participleThe keyword read is analysed to be segmented:Separator, capital and small letter change and digital alphabet change.For example, it is " Customer " that keyword Customer-username can segment according to separator" username ";Keyword CustomerSaveKeyUsername can become according to capital and small letterChanging participle is:" Customer ", " Save ", " Key " and " Username ";KeywordIt is " abc ", " 123 " and " 1 " that " abc1231 " can change participle according to digital alphabet.Here, global user identity proves (UID) class, and a kind of keyword (Key) can be used as singleSolely record.
It should be appreciated that above-mentioned detecting step prefix list when performing first time is sky, now in advanceThe prefix surveyed in the miss prefix list of prefix, prediction prefix is added to prefix list afterwards,Whether then follow-up participle can hit prefix in perform detection step with identification prediction prefixPrefix in list.
At step 420, will at the end of analytical procedure is performed to the keyword of predetermined numberPrefix of the hit mark less than 2 times is moved in individual Hash table as individual prefix in prefix listUntil all keywords in Redis databases are analyzed.
In the present embodiment, at the end of the keyword recognition of each group of predetermined number, can incite somebody to actionHit mark is moved in individual Hash table less than the prefix of 2 times as individual prefix, so as toAfter all keywords in Redis databases complete identification, step 430 is performed to be updatedPrefix list afterwards.The random character that can so avoid some programs from directly being generated by algorithmString is added in prefix list, causes the long detection for influenceing detecting step of queue in prefix listEfficiency.
In step 430, a pair key corresponding with individual prefix re-executes analytical procedure to obtainPrefix list after to renewal.
In the present embodiment, after all keywords in Redis databases complete identification,After the keyword in individual Hash table can be taken out one by one and perform analytical procedure to obtain renewalPrefix list.Herein, can also be according to length one by one by solely in order to improve recognition efficiencyKeyword in vertical Hash table takes out and performs analytical procedure.
In step 440, the keyword of the prefix in the prefix list after hit renewal is obtainedQuantity and memory space.
In the present embodiment, based on the prefix list after the renewal obtained in step 430, traversalAll keyword Key in Redis, obtain hit prefix list in prefix Key andKey value Value, and record the memory headroom that each Key and Value take.
In step 450, the keyword of the prefix in prefix list after being updated according to hitQuantity and memory space, optimize the data store organisation of Redis databases.
In the present embodiment, based on the prefix list after the hit renewal obtained in step 440In prefix keyword quantity and memory space, optimize Redis databases data storageStructure.It is exemplary, can be according to the prefix in the prefix list after the hit renewal of acquisitionThe quantity and memory space of keyword, the load pressure of caching system is prompted, to be loadedIt is balanced;The keyword of the prefix in prefix list after can also being updated according to the hit of acquisitionQuantity and memory space, optimize data storage, such as delete junk data or remove duplicate dataDeng.
Figure 4, it is seen that compared with embodiment corresponding to Fig. 2, it is excellent in the present embodimentThe flow 400 for changing the method for caching Redis data store organisation highlights pair and individual prefixThe step of corresponding key re-executes analytical procedure with prefix list after being updated.Thus,The scheme of the present embodiment description can improve the prefix in the prefix list after identification hit renewalThe efficiency of keyword, provided more accurately to optimize the data store organisation of Redis databasesData.
With further reference to Fig. 5, as the realization to method shown in above-mentioned each figure, the application providesA kind of one embodiment of the device for the data store organisation for optimizing key value database.Here stillBy taking Redis databases as an example, the device embodiment is corresponding with the embodiment of the method shown in Fig. 2,The device specifically can apply in various electronic equipments.
As shown in figure 5, the device 500 of the optimization Redis of the present embodiment data store organisationIncluding:Analytic unit 510, acquiring unit 520 and optimization unit 530.
Wherein, analytic unit 510, for reading the key of predetermined number from Redis databasesWord simultaneously performs analytical procedure until having analyzed in Redis databases to each keyword of readingAll keywords, analytical procedure includes:The keyword read is analyzed to be segmented, to dividingWord perform detection step, detecting step include:Last participle is deleted to obtain predicting prefix,Whether identification prediction prefix hits the prefix in prefix list, if hit, by prefix listThe number that is hit of prefix add 1, if miss, prediction prefix is added to prefix list,And whether identification prediction prefix is single participle, if so, then being held to next keyword of readingRow analytical procedure, if it is not, then to the participle perform detection step in prediction prefix.
Acquiring unit 520, for obtaining the quantity for the keyword for hitting the prefix in prefix listAnd memory space.
Optimize unit 530, the quantity for the keyword of the prefix in hit prefix listAnd memory space, the data store organisation of optimization Redis databases.
In some optional implementations of the present embodiment, acquiring unit is further used for:To precedingSew keyword corresponding to prefix of the number less than 2 times is hit in list and re-execute analysis stepSuddenly, the prefix list after being updated;Obtain the prefix in the prefix list after hit renewalThe quantity and memory space of keyword;And optimization unit is further used for:Updated according to hitThe quantity and memory space of the keyword of the prefix in prefix list afterwards, optimize Redis dataThe data store organisation in storehouse.
In some optional implementations of the present embodiment, analytic unit is further used for:FromThe keyword of predetermined number is read in Redis databases and each keyword to reading performsAnalytical procedure, at the end of analytical procedure is performed to the keyword of predetermined number, by prefix listMiddle prefix of the hit mark less than 2 times is moved in individual Hash table as individual prefix until having dividedAnalyse all keywords in Redis databases;Acquiring unit is further used for:Pair with independence beforeSew corresponding key and re-execute analytical procedure with the prefix list after being updated, obtain hitThe quantity and memory space of the keyword of the prefix in prefix list after renewal;And optimization is singleMember is further used for:The quantity of the keyword of the prefix in prefix list after being updated according to hitAnd memory space, the data store organisation of optimization Redis databases.
In some optional implementations of the present embodiment, analytic unit is used for from Redis dataThe keyword of predetermined number is read in storehouse to be included:Analytic unit be used for from Redis examples and/orThe keyword of predetermined number is read in Redis memory image files.
In some optional implementations of the present embodiment, analytic unit is used to analyze the pass readKey word is included with obtaining participle:Analytic unit is used for what is read according to following one or more analysesKeyword is to be segmented:Separator, capital and small letter change and digital alphabet change.
In some optional implementations of the present embodiment, optimization module is further used for:According toThe quantity and memory space of the keyword of the prefix in prefix list are hit, presents and optimizes to userThe option of data store organisation;In response to receiving selection of the user to option, optimize Redis numbersAccording to the data store organisation in storehouse.
It will be understood by those skilled in the art that the data store organisation of above-mentioned optimization key value databaseDevice 500 also include some other known features, such as processor, memory etc., in order toEmbodiment of the disclosure is unnecessarily obscured, these known structures are not shown in Figure 5.
It should be appreciated that all units described in device 500 with reference to figure 2, Fig. 3 and Fig. 4 with retouchingEach step in the method stated is corresponding.Thus, above with respect to the number of optimization key value databaseThe operation and feature described according to the method for storage organization is equally applicable to device 500 and wherein includedUnit, will not be repeated here.Corresponding units in device 500 can be with the list in serverMember cooperates to realize the scheme of the embodiment of the present application.
Below with reference to Fig. 6, it illustrates suitable for for realizing the terminal device of the embodiment of the present applicationOr the structural representation of the computer system 600 of server.
As shown in fig. 6, computer system 600 includes CPU (CPU) 601, itsCan according to the program being stored in read-only storage (ROM) 602 or from storage part 608The program that is loaded into random access storage device (RAM) 603 and perform various appropriate actionsAnd processing.In RAM 603, also it is stored with system 600 and operates required various program sumsAccording to.CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/ output (I/O) interface 605 is also connected to bus 604.
I/O interfaces 605 are connected to lower component:Importation 606 including keyboard, mouse etc.;Including cathode-ray tube (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.Output par, c 607;Storage part 608 including hard disk etc.;And including such as LAN card,The communications portion 609 of the NIC of modem etc..Communications portion 609 is via such asThe network of internet performs communication process.Driver 610 is also according to needing to be connected to I/O interfaces605.Detachable media 611, such as disk, CD, magneto-optic disk, semiconductor memory etc.,Be arranged on as needed on driver 610, in order to the computer program that reads from it according toNeed to be mounted into storage part 608.
Especially, in accordance with an embodiment of the present disclosure, can be with above with reference to the process of flow chart descriptionIt is implemented as computer software programs.For example, embodiment of the disclosure includes a kind of computer journeySequence product, it includes being tangibly embodied in the computer program on machine readable media, computerProgram bag contains the program code for being used for the method shown in execution flow chart.In such embodiments,The computer program can be downloaded and installed by communications portion 609 from network, and/or fromDetachable media 611 is mounted.
Flow chart and block diagram in accompanying drawing, it is illustrated that according to the system of the various embodiments of the application,Architectural framework in the cards, function and the operation of method and computer program product.This pointOn, each square frame in flow chart or block diagram can represent a unit, program segment or codeA part, the part of unit, program segment or code, which includes, one or more to be used to realizeThe executable instruction of defined logic function.It should also be noted that at some as the realization replacedIn, the function of being marked in square frame can also be with different from the order marked in accompanying drawing generation.For example, two square frames succeedingly represented can essentially perform substantially in parallel, they are sometimesIt can also perform in the opposite order, this is depending on involved function.It is also noted thatThe group of each square frame and block diagram in block diagram and/or flow chart and/or the square frame in flow chartClose, function or the special hardware based system of operation can be realized as defined in execution,Or it can be realized with the combination of specialized hardware and computer instruction.
Being described in unit involved in the embodiment of the present application can be real by way of softwareIt is existing, it can also be realized by way of hardware.Described unit can also be arranged on processingIn device, for example, can be described as:A kind of processor includes analytic unit, acquiring unit and excellentChange unit.Wherein, the title of these units is not formed to the unit in itself under certain conditionsRestriction, for example, analytic unit is also described as " reading predetermined number from databaseKeyword and analytical procedure is performed to each keyword of reading until in analytical databaseAll keywords unit ".
As on the other hand, present invention also provides a kind of nonvolatile computer storage media,The nonvolatile computer storage media can be non-easy included in device in above-described embodimentThe property lost computer-readable storage medium;Can also be individualism, without non-volatile in supplying terminalProperty computer-readable storage medium.Above-mentioned nonvolatile computer storage media is stored with one or moreIndividual program, when one or more program is performed by an equipment so that equipment:From dataIt is straight to perform analytical procedure for the keyword of reading predetermined number and each keyword to reading in storehouseAll keywords into analytical database, analytical procedure include:Analyze the keyword readTo be segmented, to segmenting perform detection step, detecting step includes:Delete last pointFor word to obtain predicting prefix, whether identification prediction prefix hits the prefix in prefix list, if lifeIn, then the number that is hit of the prefix in prefix list is added 1, will prediction if missPrefix is added to prefix list, and whether identification prediction prefix is single participle, if so, then rightThe next keyword read performs analytical procedure, if it is not, then being held to the participle in prediction prefixRow detecting step;Obtain the quantity and memory space of the keyword of the prefix in hit prefix list;The quantity and memory space of the keyword of prefix in hit prefix list, optimize databaseData store organisation.
Above description is only the preferred embodiment of the application and saying to institute's application technology principleIt is bright.It will be appreciated by those skilled in the art that invention scope involved in the application, and it is unlimitedIn the technical scheme that the particular combination of above-mentioned technical characteristic forms, while it should also cover and not depart fromIn the case of inventive concept, it is combined and shape by above-mentioned technical characteristic or its equivalent featureInto other technical schemes.Such as features described above has with (but not limited to) disclosed hereinThe technical scheme that the technical characteristic of similar functions is replaced mutually and formed.

Claims (12)

  1. The keyword of predetermined number is read from database and each keyword to reading performsAnalytical procedure is until all keywords in analytical database, the analytical procedure include:PointThe keyword read is analysed to be segmented, to the participle perform detection step, the detection stepSuddenly include:Last participle is deleted to obtain predicting prefix, whether identifies the prediction prefixThe prefix in prefix list is hit, if hit, the prefix in the prefix list is orderedMiddle number adds 1, if miss, the prediction prefix is added into the prefix list, andIdentify whether the prediction prefix is single participle, if so, then to next keyword of readingAnalytical procedure is performed, if it is not, then performing the detecting step to the participle in the prediction prefix;
  2. Analytic unit, for reading the keyword of predetermined number from database and to the every of readingOne keyword performs analytical procedure until all keywords in analytical database, described pointAnalysis step includes:The keyword read is analyzed to be segmented, to the participle perform detection stepSuddenly, the detecting step includes:Last participle is deleted to obtain predicting prefix, identifies instituteState and predict whether prefix hits the prefix in prefix list, if hit, by the prefix listIn the number that is hit of prefix add 1, if miss, the prediction prefix is added to institutePrefix list is stated, and identifies whether the prediction prefix is single participle, if so, then to readingNext keyword perform analytical procedure, if it is not, then to it is described prediction prefix in participle holdThe row detecting step;
CN201610305828.7A2016-05-102016-05-10Method and device for optimizing data storage structure of key value databaseActiveCN107357794B (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN201610305828.7ACN107357794B (en)2016-05-102016-05-10Method and device for optimizing data storage structure of key value database

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN201610305828.7ACN107357794B (en)2016-05-102016-05-10Method and device for optimizing data storage structure of key value database

Publications (2)

Publication NumberPublication Date
CN107357794Atrue CN107357794A (en)2017-11-17
CN107357794B CN107357794B (en)2020-06-05

Family

ID=60271375

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN201610305828.7AActiveCN107357794B (en)2016-05-102016-05-10Method and device for optimizing data storage structure of key value database

Country Status (1)

CountryLink
CN (1)CN107357794B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN110413546A (en)*2019-06-192019-11-05平安科技(深圳)有限公司Date storage method, device and computer readable storage medium based on redis
WO2020000734A1 (en)*2018-06-282020-01-02平安科技(深圳)有限公司Space management method and apparatus for storage object, and computer apparatus and storage medium
CN111125095A (en)*2019-11-262020-05-08北京文渊佳科技有限公司 Method, device, electronic device and medium for adding data prefix
CN111339736A (en)*2020-02-182020-06-26江苏满运软件科技有限公司Method for adding prefix name, configuration acquisition method and device and electronic equipment
CN112148554A (en)*2020-09-142020-12-29北京金和网络股份有限公司Method and device for calculating occupation size of redis service data in real time

Citations (5)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN101282313A (en)*2008-05-222008-10-08北京航空航天大学 An E-mail System Used in Electronic Meeting Auxiliary System
US20130138646A1 (en)*2010-04-272013-05-30Emin Gun SirerSystem and methods for mapping and searching objects in multidimensional space
CN105373541A (en)*2014-08-222016-03-02博雅网络游戏开发(深圳)有限公司Processing method and system for data operation request of database
CN105491116A (en)*2015-11-262016-04-13广州华多网络科技有限公司Cross-window data submitting method and system
US20160110292A1 (en)*2014-10-212016-04-21Samsung Electronics Co., Ltd.Efficient key collision handling

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN101282313A (en)*2008-05-222008-10-08北京航空航天大学 An E-mail System Used in Electronic Meeting Auxiliary System
US20130138646A1 (en)*2010-04-272013-05-30Emin Gun SirerSystem and methods for mapping and searching objects in multidimensional space
CN105373541A (en)*2014-08-222016-03-02博雅网络游戏开发(深圳)有限公司Processing method and system for data operation request of database
US20160110292A1 (en)*2014-10-212016-04-21Samsung Electronics Co., Ltd.Efficient key collision handling
CN105491116A (en)*2015-11-262016-04-13广州华多网络科技有限公司Cross-window data submitting method and system

Cited By (8)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
WO2020000734A1 (en)*2018-06-282020-01-02平安科技(深圳)有限公司Space management method and apparatus for storage object, and computer apparatus and storage medium
CN110413546A (en)*2019-06-192019-11-05平安科技(深圳)有限公司Date storage method, device and computer readable storage medium based on redis
CN110413546B (en)*2019-06-192024-03-12平安科技(深圳)有限公司Redis-based data storage method, device and computer readable storage medium
CN111125095A (en)*2019-11-262020-05-08北京文渊佳科技有限公司 Method, device, electronic device and medium for adding data prefix
CN111125095B (en)*2019-11-262023-11-10北京文渊佳科技有限公司 Methods, devices, electronic equipment and media for adding data prefixes
CN111339736A (en)*2020-02-182020-06-26江苏满运软件科技有限公司Method for adding prefix name, configuration acquisition method and device and electronic equipment
CN111339736B (en)*2020-02-182023-09-01江苏满运软件科技有限公司Method for adding prefix name, configuration acquisition method, device and electronic equipment
CN112148554A (en)*2020-09-142020-12-29北京金和网络股份有限公司Method and device for calculating occupation size of redis service data in real time

Also Published As

Publication numberPublication date
CN107357794B (en)2020-06-05

Similar Documents

PublicationPublication DateTitle
CN106446228B (en)Method and device for collecting and analyzing WEB page data
CN105183912B (en)Abnormal log determines method and apparatus
CN107357794A (en)Optimize the method and apparatus of the data store organisation of key value database
CN104077407B (en)A kind of intelligent data search system and method
US10496696B2 (en)Search method and apparatus
CN107908615A (en)A kind of method and apparatus for obtaining search term corresponding goods classification
CN107908616B (en)Method and device for predicting trend words
CN112286815A (en)Interface test script generation method and related equipment thereof
CN117593096B (en)Intelligent pushing method and device for product information, electronic equipment and computer medium
CN103324742A (en)Method and equipment for recommending keywords
CN111400361A (en)Data real-time storage method and device, computer equipment and storage medium
CN112632981B (en)New word discovery method and device
CN113297471B (en)Data object tag generation method, data object searching device and electronic equipment
CN104580109B (en)Generation clicks the method and device of identifying code
CN109754295A (en)Method and apparatus for output information
JP6025487B2 (en) Forensic analysis system, forensic analysis method, and forensic analysis program
CN106919593B (en) A search method and device
CN113297345B (en)Analysis report generation method, electronic equipment and related product
CN107368484A (en)Compression method and device, the acquisition methods and device of the static resource file of webpage
WO2018208412A1 (en)Detection of caption elements in documents
US20130230248A1 (en)Ensuring validity of the bookmark reference in a collaborative bookmarking system
US20240233427A1 (en)Data categorization using topic modelling
CN109241428B (en) Method, device, server and storage medium for determining user gender
US20170242991A1 (en)System and method for resolving user identification
JP2018181121A (en) Analyzer, analysis program and analysis method

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination
GR01Patent grant
GR01Patent grant

[8]ページ先頭

©2009-2025 Movatter.jp