CN101853299B - Image searching result ordering method based on perceptual cognition - Google Patents

Abstract

The invention discloses an image searching result ordering method based on perceptual cognition, which mainly orders images results based on the objective characteristics and the subjective perceptual cognition of images. Based on the understanding to the images, the characteristics of the images are extracted to obtain the similarity and a display method is laid out based on the user experience. Under the circumstance that the invention cannot be accurately expressed, a user can better display the shopping need on a platform, thereby reducing the time for the user to search a commodity and more effectively promoting a network commodity transaction. Simultaneously, the image searching result ordering method based on perceptual cognition can promote the development of the electronic shopping platform so that China's electronic commerce network platform is more diversified.

Description

A kind of image searching result ordering method based on perceptual recognition

Technical field

The present invention relates to the picture search technical field, relate in particular to a kind of image searching result ordering method based on perceptual recognition.

Background technology

There are several typical search engines in the market, comprise the search dog of Baidu, Google, Sohu and the Bing of Microsoft.Above-mentioned search engine is fit to various users towards text, and has captured the market of text search basically.

But text search engine also has some deficiency and defective.When people will search for some content that can't definitely describe; Perhaps need search plain content and contain subjective concept; Or the data (various multi-medium datas such as audio frequency, video, image, 3D grid) that need search and known format are very similar as a result the time, and text search has just shown its deficiency.For this reason, some search engines based on image have appearred on the market.

Search engine based on image need be imported sample figure, from database, searches the result similar with the characteristics of image of sample figure then.Such as,Www.tinyeye.com, www.like.com just is based on the example of image retrieval (being designated as CBIR:Content Based Image Retrieval).Most of search engine based on image is the basis with technology such as picture material understanding and pattern-recognitions, and towards various images.

Existing C BIR has following weak point: for the CBIR system of dress ornament class, existing application only relies on visual feature of image to retrieve, and does not consider the purchasing environment of people in reality, lacks the actual experience sense.In addition, the view data in the existing system is all relatively unified, and data volume is also few.

Along with the explosive growth of Various types of data on the internet, it is more and more that people seek the required time of target product.In addition, though there are various powerful text search engines, for the dress ornament series products, the user usually can't utilize literal accurate description demand.

Summary of the invention

The objective of the invention is to deficiency, a kind of image searching result ordering method based on perceptual recognition is provided to prior art.The present invention utilizes master drawing to describe user's demand, in the understanding based on picture material, helps the user to search out target product (mainly being the dress ornament series products) fast.

The present invention mainly sorts to image result based on the objective characteristics and the subjective sensibility cognition of image.On basis to image understanding, extract characteristics of image, obtain similarity, and display mode is carried out layout based on user experience.This mode not only can in time find user's target product, and can give the great visual impact of user, actively guides the user to get into the shopping link.This is that text retrieval can't realize, is a kind of novel electronic shopping guide's platform.

In order to set up a kind of like this convenience intelligent shopping guide platform intuitively, the present invention takes following steps as technical scheme.

1) at first, sets up the raw data base that contains great amount of images.

In order to set up image data base, need utilize web crawlers to go to the various websites of containing dress ornament class image to grasp raw data.

2) to every image in the storehouse, obtain the precise region at target place in the image, be designated as MASK zone (masked areas: be designated as MASK).

In order to obtain the MASK zone, need a kind of automanual target localization subsystem of exploitation, be used for confirming the Position Approximate of dress ornament, and utilize image Segmentation Technology to obtain zone accurately on this basis at image.

3) the MASK zone according to every image obtains various characteristics of image.

To dress ornament class image, the parameter that can be used for the characteristic statement has color, shape, texture and pattern etc.Different types of clothes possibly obtain different character.Do not need shape facility such as T-shirt, but need pattern characteristics.

4) set up the integrated data base that contains image and characteristic.

Entire database is made up of raw image data storehouse and property data base.Store for ease and visit, need be with view data and characteristic fragmented storage.Whenever newly-increased image all need extract its characteristic, and deposit it in property data base in the storehouse.Image of every deletion need be deleted original image and its characteristic simultaneously.

5) build the B/S structure platform, the retrieval service based on sample figure is provided to the user.

Integrated data base is placed on server, and client is set up a terrace at entrance, makes things convenient for the user to upload image, perhaps from the storehouse, selects image to retrieve as sample figure.Server end is according to characteristics such as the color of image, shape, local pattern, and according to image series more approaching with sample figure in the similarity return data storehouse, final result for retrieval is presented at client.

6) among the result for retrieval figure, the result is sorted according to the perception similarity.

In the Figure List as a result that retrieval obtains based on master drawing,, carry out the displaying of figure as a result with the objective characteristics (color characteristic, shape or pattern characteristics etc.) and the similarity of subjective characteristics (product style etc.) according to layout type.Contain information such as commodity price, the link of corresponding network businessman as a result among the figure simultaneously.Click figure as a result, can this as a result figure carry out the retrieval of a new round as input figure.

The invention has the beneficial effects as follows:, under the situation that text can't accurately be expressed, can find the target commodity sooner, more accurately in conjunction with the image similarity of subjective characteristics and objective characteristics as a kind of novel electronic shopping guide platform.The user can show the shopping need of oneself better on this platform, reduce searching the time of commodity, facilitates the network commodity transaction more efficiently.Simultaneously, the present invention will promote the development of novel electron shopping platform, make the diversification more of China Electronic Data Interchange network network platform.

Description of drawings

Fig. 1 is system framework figure;

Fig. 2 is that color characteristic extracts process flow diagram;

Fig. 3 is the Shape Feature Extraction synoptic diagram;

Fig. 4 is the B/S configuration diagram;

Fig. 5 is local matching module process flow diagram;

Fig. 6 is the Search Results display effect synoptic diagram of customer terminal webpage.

Embodiment

With the retrieval of dress ornament class image be shown as example, the present invention is done further detailed explanation below in conjunction with accompanying drawing.The operation that the present invention relates to can be comprehensively for shown in the following table, and that the framework of total system and flow process are seen is shown in Figure 1.

Image searching result ordering method based on perceptual recognition of the present invention may further comprise the steps:

1.1) before design of graphics is as feature database, adopt the target automatic positioning method, obtain the masked areas of warehouse-in image.

Visible by Fig. 1, the target extraction module comprises that network data grasps, tentatively filters four steps such as submodule, target localization submodule, image mask extraction submodule.The network data grabbing module is utilized the web crawlers robot, collects relevant dress ornament image from the internet.But the reptile robot only grasps according to the judgement of picture format, and therefore not all image that downloads to all is required dress ornament class image.Preliminary function of filtering submodule is deleted some tangible non-dress ornament class images exactly, and filtering policy comprises: form filters, and promptly only downloads the image of specific format.Size filtered is promptly according to some useless images of data filter such as image size, length breadth ratios.Image attributes filters, and removes all achromatic images.Because all kinds of images do not have standards and norms on the network, very disunity.In therefore a lot of images, do not have fixing background, possibly contain quite a few clothes or dress ornament class article in the image, possibly contain model or the like in the image.The target localization submodule mainly is used for confirming the Position Approximate at place, target area such as clothes.

In the target localization submodule, need classify to raw image data, take diverse ways to position to different classification.Present classification has:

● clothes tiling type: photographer can be placed on clothes with the clothes color during based on tiling has the hypothesis under the background of discrimination; Therefore adopt big Tianjin method (OSTU algorithm) directly to carry out binary conversion treatment; Connected region information among the analysis of binary figure is then finally confirmed the rational position of target.Such locating effect is more satisfactory, and can directly obtain mask MASK data, and the masked areas of having omitted the back is obtained the processing of submodule.

● clothes model class: the model is all arranged, the algorithm that can adopt people's face to detect, the approximate region of acquisition clothes in the many clothes image.

● clothes lattice class: the lattice that at first detects clothes is interval, and then diverse ways is adopted in each interval respectively.

● other classification: except other classification of above-mentioned classification.

The target localization result is a rectangular frame, except object, also might have other object or background in the frame.Therefore, need obtain the precise region of object, this just needs mask to extract.It is on the basis of target localization submodule that the image mask extracts submodule, obtains the exact position of target in the image.Adopt convergence algorithm at present based on least energy.

1.2) obtain the masked areas of image after, extract the color characteristic of image.

The method for distilling of color characteristic is as shown in Figure 2.Step is following:

● color quantizing: with 8 in each passage totally 256 grades be quantified as 16 grades, totally 4096 grades in three passages of Red Green Blue RGB, i.e. 4096 grid Bin.

● color cluster:, obtain color histogram according to the distribution of color after quantizing.Getting preceding N (N=8 at present) position color is initial cluster center, utilizes K-Means to carry out color cluster.

Characteristic is preserved: the color after the final cluster is transformed into hue-saturation-brightness color space (HSV space) from RGB.The HSV space is quantified as 36000 grades, is respectively 360 grades of H values, each 10 grades of S value and V values.The proportion that hsv color after the conversion is classified and such color accounts for is saved in tag file.

1.3) obtain the masked areas of image after, obtain feature of image shape.

Shape facility obtain main employing " N collimation method ", as shown in Figure 3.Utilize the N collimation method in masked areas, weigh the ratio of every line and MASK width, with the ratio value array of N bar line as shape facility.To different dress ornament classifications, the length breadth ratio that also need obtain masked areas is as a simple shape facility.

1.4) obtain the masked areas of image after, for special category clothes such as T-shirts, obtain characteristics such as pattern.

Pattern characteristics only obtains in special defects purpose image, and its method is: at first adopt the area of the pattern automatic positioning method, estimate rectangle (RECT) zone at the place of the pattern on the T-shirt in the MASK zone roughly.For the inaccurate image in automatic location, adopt artificial picture frame to confirm the RECT zone.After confirming the pattern RECT zone on the clothes, the SIFT characteristic of obtaining area of the pattern is as pattern characteristics.

1.5) making up searching database, entire database is made up of original image and tag file two parts.

What make up the searching database correspondence is " ADD " operation, and this process can be referred to as " warehouse-in ".Whole in order to make " warehouse-in " process automation need be set up a whole set of flow process mechanism and handle, examines standard.As shown in Figure 1, inhomogeneity purpose dress ornament is placed in the different files catalogue, and original image institutional framework according to the rules is placed under the particular path, constitutes whole original image storehouse.And image of every warehouse-in just obtains its various visual signatures, and in tag file, increases response record.Different character is recorded in the different character file.Because some characteristic is complicated, possibly also need a plurality of files storage feature data respectively.

2) setting up shopping guide's platform with the B/S framework.

Shopping guide's platform adopts the B/S framework, and promptly the internet terminal user can pass through terminal browser access shopping guide platform.Service end needs multiple servers simultaneously, comprises application server, search engine server, database server and file server, and whole framework is as shown in Figure 4.Wherein, application server provides external web-page interface, supplies user capture, and collects user's request.After the user sends searching request, application server will be handed to the image engine server to request, obtain similarity information by the latter, and return result for retrieval.In the processing procedure of whole retrieval request, also need the cooperation of image server and database server, jointly the result for retrieval image sequence is turned back to application server, and finally be presented at client browser.

Above-mentioned framework can be supported the visit that the large user measures; Each server node all can be expanded; Adopt trunking mode,, all can dispose many like application server, image engine server, file server, database server; Unification outwards provides service, can support millions other day user visit capacity.

3.1) according to visual feature of image, result for retrieval is sorted.

When the commodity image is sorted, consider at first whether the local feature of image is similar, promptly carry out the part coupling earlier, obtain the similarity tabulation.On the basis of part coupling, carry out level then and filter, obtain two minor sorts according to characteristics such as color, shape or patterns.

Local coupling is mainly used in from database, to retrieve and contains fully, perhaps contains the image of most of input master drawing.The whole algorithm step is as shown in Figure 5, and is specific as follows:

● the characteristic of every image in the training image database generates N sight word (Visual Words).At first extract the SIFT characteristic in all databases, adopt cascade K-Means algorithm that the SIFT characteristic is carried out cluster then, generate N characteristic center, and this is gathered as sight word.

● for follow-up SIFT characteristic matching, obtain the Hamming code of each SIFT characteristic, and preserve together with the SIFT characteristic.

● utilize MSER (Most Stable External Region: the most steady perimeter) algorithm, obtain the MSER characteristic of every image in the image data base.

● MSER and SIFT characteristic are bound.If the corresponding zone of certain MSER characteristic has no the SIFT characteristic, then remove this MSER characteristic.Otherwise the SIFT feature set that contains with the corresponding zone of certain MSER characteristic is as the essential characteristic unit of subsequent characteristics retrieval.

● before retrieving, need to preserve above-mentioned SIFT feature database, corresponding Hamming code set, and sight word set.

● when retrieving, at first obtain the binding characteristic of MSER and the SIFT of sample figure.Add up each then and bind the pairing sight word set of characteristic, and find the database images that contains same sight word, weigh matching degree between the two according to each sight word in the set.Each binding characteristic among the sample figure is implemented above-mentioned steps, and sets up a voting mechanism, the record matching degree.

● the process of voting mechanism is following: each sight word that SIFT shone upon is inquiry in the sight word set all; To the marking of voting of the binding characteristic in the image that contains this sight word that inquires, voting results are placed in the interim result queue, and voting results are enclosed the numbering of binding characteristic; Be used for arrangement to voting results; After all sight word have all been inquired about, put interim result queue in order, SIFT binds characteristic to an of image; Only keep a highest ticket of score, the ticket of repetition is all deleted; Result after the arrangement is deposited in the ballot formation.

● the arrangement voting results, the score of adding up every image sorts to image by mark, and the result writes back the ballot formation.

In order to obtain final result for retrieval, adopt the level filtering policy.At first utilize the local feature matching process to carry out preliminary screening, the result after the screening is sent into CF characteristic (perhaps pattern characteristics) module carry out similarity coupling further.And final result returned to client.

3.2) display page carries out layout, similarity is sorted.

All result for retrieval will return to client, and be presented at client browser.Display mode can have multiple different layout.Shown in Figure 6ly be wherein a kind of, client shows with clinodiagonal as distinguishing line, on directions X and Y direction (is initial point with the upper left corner), carries out the displaying of figure as a result according to the similarity of color characteristic and shape facility (pattern characteristics, local feature) respectively.Contain information such as commodity price, the link of corresponding network businessman as a result among the figure simultaneously.Click figure as a result, can this as a result figure carry out the retrieval of a new round as input figure.

3.3) in the display page, every as a result figure contain multiple attributes such as commodity price, businessman's link, rate of exchange link simultaneously.

In page; For more selection is provided to the user; The guiding client checks the information that commodity are relevant quickly, and the comparison between the commodity, at each display page as a result; Except providing as a result the figure, the information such as businessman's link and rate of exchange link of commodity price information, commodity are provided (above or below) around the figure as a result also.

After research user's shopping custom and user's shopping online experience, will carry out specific layout arrangement to result images and relevant information thereof, make that the user is easier, more convenient, buy the commodity of wanting faster.Final purpose is in order to facilitate network trading fast.

Claims (1)

1. the image searching result ordering method based on perceptual recognition is characterized in that, comprises the steps:

(1) design of graphics is as feature database;

1.1) before design of graphics is as feature database, obtain the masked areas of warehouse-in image in advance;

1.2) obtain the masked areas of image after, obtain the color characteristic of image;

1.3) obtain the masked areas of image after, obtain feature of image shape;

1.4) obtain the masked areas of image after, for T-shirt, obtain pattern characteristics;

1.5) making up searching database, entire database is made up of original image and tag file two parts;

(2) set up shopping guide's platform with the B/S framework in client, the user can select in the storehouse image or other image to retrieve as input, and result for retrieval returns client;

(3) similarity based on perceptual recognition shows result for retrieval;

3.1) sort according to the synthesis result of the color of image, shape, pattern characteristics;

3.2) in the display page, can select respectively to sort according to the similarity of color, shape, pattern characteristics through button;

3.3) in the display page, every as a result figure contain multiple attributes such as commodity price, businessman's link, rate of exchange link simultaneously;

Wherein, in the said step (1.1), the acquisition methods of said image masked areas is: adopt the target automatic positioning method, estimate the rectangular region at target object place in the image roughly; For the inaccurate image in automatic location, adopt artificial picture frame to confirm rectangular region; After confirming rectangular region, utilize the non-systematicness of image segmentation algorithm acquisition target accurately regional, i.e. masked areas;

In the said step (1.2), the color characteristic acquisition methods of image is: at first Red Green Blue is quantized, form limited grid; Distribution of color according to after quantizing obtains color histogram; L position color is an initial cluster center before getting, and utilizes the K-mean algorithm to carry out color cluster; Color after the final cluster is transformed into the hue-saturation-brightness color space from Red Green Blue; The hue-saturation-brightness color space is quantified as the M level, is respectively tone value M₁Level, each M of intensity value and brightness value₂Level; The proportion that hue-saturation-brightness color space color classification after changing the most at last and such color account for is saved in tag file;

In the said step (1.3), the feature of image shape acquisition methods is: utilize the N collimation method in masked areas, weigh the ratio of every line and masked areas width, with the ratio value array of N bar line as shape facility; And for the image of case and bag, shape facility also comprises length breadth ratio;

In the said step (1.4), the pattern characteristics acquisition methods of image is: at first adopt the target automatic positioning method, estimate the rectangular region at pattern place in the masked areas roughly; For the inaccurate image in automatic location, adopt artificial picture frame to confirm rectangular region; After confirming rectangular region, the eigentransformation SIFT characteristic of convergent-divergent, invariable rotary of obtaining area of the pattern is as pattern characteristics;

In the said step (1.5), the view data in the entire database derives from network, and every image need obtain various characteristics separately, and deposit tag file in before warehouse-in; Adopt segmented mode memory image characteristic, read in the characteristic of all images of a segmentation at every turn, accelerate the retrieval rate in later stage;

In the said step (2), searching step is: import master drawing, at first obtain the masked areas of master drawing, obtain the characteristic of master drawing then in masked areas; The characteristic of image in master drawing characteristic and the database is compared, and N opens figure as a result before returning; Master drawing can be in the storehouse, also can be that user oneself uploads;

In the said step (3), in the Figure List as a result that retrieval obtains based on master drawing, carry out the displaying of figure as a result according to the perception similarity; Contain commodity price, the link of corresponding network businessman as a result among the figure simultaneously; Click figure as a result, can this as a result figure carry out the retrieval of a new round as input figure.

Images