技术领域technical field
本发明涉及计算机领域，尤其涉及一种用于评价搜索结果的技术。The invention relates to the field of computers, in particular to a technique for evaluating search results.
背景技术Background technique
在现有技术中，对搜索结果的评价主要依赖于用户对搜索结果的点击行为，而未考虑用户在搜索过程中不需对所述搜索结果进行点击操作即可满足其搜索需求的情形，例如，用户仅通过搜索结果页面展示的搜索结果摘要即可满足查询需求，而无需点击各搜索结果项的链接以进入其所对应的页面。由于忽略对用户无点击操作的评价，从而导致对搜索结果评价的不够准确。In the prior art, the evaluation of search results mainly depends on the user's click behavior on the search results, without considering the situation that the user can satisfy his search needs without clicking on the search results during the search process, such as , the user can satisfy the query requirement only through the search result summary displayed on the search result page, without clicking the link of each search result item to enter the corresponding page. Due to ignoring the evaluation of the user's non-click operation, the evaluation of the search results is not accurate enough.
因此，如何有效并准确地评价搜索结果，成为目前亟待解决的问题之一。Therefore, how to effectively and accurately evaluate search results has become one of the problems to be solved urgently.
发明内容Contents of the invention
本发明的目的是提供一种用于评价搜索结果的方法与设备。It is an object of the present invention to provide a method and device for evaluating search results.
根据本发明的一个方面，提供了一种计算机实现的用于评价搜索结果的方法，该方法包括以下步骤：According to one aspect of the present invention, there is provided a computer-implemented method for evaluating search results, the method comprising the steps of:
a.获取一个或多个用户对待评价的基于相同查询序列的搜索结果的非点击操作信息；a. Obtain non-click operation information of one or more users to be evaluated based on the search results of the same query sequence;
b.根据所述非点击操作信息，对所述搜索结果进行评价。b. Evaluate the search results according to the non-click operation information.
根据本发明的另一方面，还提供了一种用于评价搜索结果的设备，该设备包括：According to another aspect of the present invention, there is also provided a device for evaluating search results, the device comprising:
操作信息获取装置，用于获取一个或多个用户对待评价的基于相同查询序列的搜索结果的非点击操作信息；An operation information acquisition device, configured to acquire non-click operation information of one or more users to be evaluated based on the search results of the same query sequence;
评价装置，用于根据所述非点击操作信息，对所述搜索结果进行评价。An evaluating device, configured to evaluate the search result according to the non-click operation information.
与现有技术相比，本发明通过获取用户对待评价的基于相同查询序列的搜索结果的非点击操作信息，如是否输入新的查询序列、对搜索结果项的划选操作等，并基于预定评价规则对该搜索结果进行评价，将用户对搜索结果执行无点击操作时即可满足搜索需求的情形包含至评价标准中，从而为提高搜索结果评价的准确性提供更为可靠的依据。Compared with the prior art, the present invention obtains the non-click operation information of the search results based on the same query sequence to be evaluated by the user, such as whether to input a new query sequence, the selection operation of the search result items, etc., and based on the predetermined evaluation The rules evaluate the search results, and include the situation that the user can satisfy the search needs without clicking on the search results into the evaluation criteria, so as to provide a more reliable basis for improving the accuracy of the search result evaluation.
附图说明Description of drawings
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本发明的其它特征、目的和优点将会变得更明显：Other characteristics, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
图1示出根据本发明一个方面的用于评价搜索结果的设备示意图；Figure 1 shows a schematic diagram of a device for evaluating search results according to one aspect of the present invention;
图2示出根据本发明另一个方面的用于评价搜索结果的方法流程图。FIG. 2 shows a flowchart of a method for evaluating search results according to another aspect of the present invention.
附图中相同或相似的附图标记代表相同或相似的部件。The same or similar reference numerals in the drawings represent the same or similar components.
具体实施方式Detailed ways
下面结合附图对本发明作进一步详细描述。The present invention will be described in further detail below in conjunction with the accompanying drawings.
图1示出根据本发明一个方面的评价搜索结果的设备示意图。其中，评价设备1包括操作信息获取装置111和评价装置112。FIG. 1 shows a schematic diagram of an apparatus for evaluating search results according to one aspect of the present invention. Wherein, the evaluation device 1 includes an operation information acquisition device 111 and an evaluation device 112 .
在此，评价设备1可以是一个网络设备或位于其中的功能模块，其中，网络设备包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云，在此，云由基于云计算(Cloud Computing)的大量计算机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个超级虚拟计算机；评价设备1也可以是用户设备，其包括但不限于任何一种可与用户通过键盘、遥控器、触摸板、或声控设备进行人机交互的电子产品，例如计算机、智能手机、PDA、游戏机、或IPTV等。Here, the evaluation device 1 may be a network device or a functional module located therein, wherein the network device includes but not limited to a computer, a network host, a single network server, a set of multiple network servers or a cloud composed of multiple servers. , the cloud is composed of a large number of computers or network servers based on cloud computing (Cloud Computing), wherein cloud computing is a kind of distributed computing, a super virtual computer composed of a group of loosely coupled computer sets; the evaluation device 1 can also be User equipment, including but not limited to any electronic product that can interact with the user through a keyboard, remote control, touch pad, or voice-activated device, such as a computer, smart phone, PDA, game console, or IPTV.
以下参照图1来对评价设备1评价搜索结果的过程进行详细描述：The process of evaluating the search results by the evaluation device 1 is described in detail below with reference to FIG. 1 :
具体地，操作信息获取装置111获取一个或多个用户对待评价的基于相同查询序列的搜索结果的非点击操作信息。Specifically, the operation information obtaining means 111 obtains the non-click operation information of the search results based on the same query sequence to be evaluated by one or more users.
在此，所述非点击操作信息包括但不限于以下任一项：Here, the non-click operation information includes but not limited to any of the following:
1)用户输入新的查询序列的操作信息；例如，在已获得搜索结果后，用户在搜索结果页面中的搜索输入框又进行输入操作，且该输入操作对应的新的查询序列与前次查询序列相关，例如两个查询序列中有部分查询关键字完全相同，则可确定用户在其搜索需求未被满足的情况下输入新的查询序列；1) The operation information of the user inputting a new query sequence; for example, after obtaining the search results, the user performs another input operation in the search input box on the search result page, and the new query sequence corresponding to the input operation is the same as the previous query Sequence correlation, for example, if some of the query keywords in two query sequences are identical, it can be determined that the user enters a new query sequence when his search needs are not met;
2)用户对搜索结果的各个搜索结果项的划选操作信息；其中，所述划选操作信息包括但不限于用户对搜索结果的各个搜索结果项进行划选操作的次数；例如，根据鼠标点下的操作相关信息，如点下的位置坐标及点下的时间信息，以及相应地释放鼠标时该鼠标释放的操作相关信息，如释放的位置坐标及释放的时间信息，以记录该等操作为一次划选操作，同时，根据该点下的位置坐标与释放的位置坐标，确定该次划选操作对应的搜索结果项，并根据该点下的时间信息与放开的时间信息，确定该次划选操作的时长；2) The user's selection operation information on each search result item of the search result; wherein, the selection operation information includes but not limited to the number of times the user performs a selection operation on each search result item of the search result; for example, according to the mouse click Information related to the operation under the click, such as the position coordinates of the click and the time information of the click, and the operation related information of the mouse when the mouse is released correspondingly, such as the position coordinates of the release and the time information of the release, so as to record such operations as A swipe operation, at the same time, according to the position coordinates under the point and the released position coordinates, determine the search result item corresponding to the swipe operation, and determine the time information according to the time information under the point and the release time information. The duration of the selection operation;
3)用户对搜索结果的各个搜索结果项的悬停操作信息；3) The hover operation information of each search result item of the search result by the user;
其中，所述用户对所述搜索结果的各个搜索结果项的悬停操作信息包括以下至少任一项：Wherein, the user's hover operation information on each search result item of the search result includes at least any of the following:
a)用户在搜索结果的各个搜索结果项上的悬停操作次数；例如，实时地获取鼠标移入搜索结果的某个搜索结果项所在区域的时间以及移出该区域的时间，当移入时间与移出时间的时间间隔超过预定时间间隔阈值时，记录为一次悬停操作，进而统计用户在该搜索结果所在搜索结果页面中的总的悬停操作次数；a) The number of times the user hovers over each search result item in the search results; for example, the time when the mouse moves into the area where a certain search result item of the search result is located and the time when the mouse moves out of the area are obtained in real time. When the time interval exceeds the predetermined time interval threshold, it is recorded as a hovering operation, and then the total number of hovering operations of the user on the search result page where the search result is located is counted;
b)用户在搜索结果的各个搜索结果项上的每次悬停操作的持续时间；例如，实时地获取鼠标移入搜索结果的某个搜索结果项所在区域的时间以及移出该区域的时间，记录移入时间与移出时间的时间间隔，即每次悬停操作的持续时间。b) The duration of each hovering operation of the user on each search result item of the search result; for example, obtain the time when the mouse moves into the area where a certain search result item of the search result is located and the time when the mouse moves out of the area in real time, and record the time when the mouse moves in The interval between time and moveout time, i.e. the duration of each hover action.
本领域技术人员应能理解上述悬停操作信息仅为举例，其他现有的或今后可能出现的悬停操作信息如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above-mentioned hovering operation information is only an example, and other existing or future hovering operation information, if applicable to the present invention, should also be included in the protection scope of the present invention, and shall be incorporated by reference included here.
本领域技术人员应能理解上述非点击操作信息仅为举例，其他现有的或今后可能出现的非点击操作信息如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above-mentioned non-click operation information is only an example, and other existing or future non-click operation information, if applicable to the present invention, should also be included in the protection scope of the present invention, and shall be cited included here.
在此，所述获取非点击操作信息的方式包括但不限于以下任一情形：Here, the methods of obtaining non-click operation information include but are not limited to any of the following situations:
1)操作信息获取装置111获取一个用户通过用户设备对待评价的基于相同查询序列的搜索结果执行的非点击操作信息；1) The operation information obtaining means 111 obtains non-click operation information performed by a user through the user equipment on the search results based on the same query sequence to be evaluated;
2)操作信息获取装置111应事件触发实时地、或周期地接收来自第三方设备发送的多个用户对待评价的基于相同查询序列的搜索结果的非点击操作信息。2) The operation information acquisition unit 111 receives the non-click operation information of the search results based on the same query sequence to be evaluated by multiple users sent from the third-party device in real time or periodically in response to an event trigger.
在一示例中，当评价设备1为网络设备时，首先，用户借助用户设备的交互装置，包括但不限于键盘、鼠标、遥控器、触摸板、或手写设备，与用户设备的浏览器软件或客户端软件进行交互。以鼠标为例，用户在用户设备的浏览器软件的搜索结果页面中进行划选操作时，该用户设备实时地获取用户对搜索结果的某一搜索结果项的划选操作，例如用户设备首先获取鼠标点下的操作相关信息，如点下的位置坐标及点下的时间信息，接着当释放该鼠标时，获取该鼠标释放的操作相关信息，如释放的位置坐标及释放的时间信息，并记录该次操作为划选操作，同时，根据该点下的位置坐标与释放的位置坐标，确定该次划选操作对应的搜索结果项，并根据该点下的时间信息与释放的时间信息，确定该次划选操作的时长；从而确定与该用户划选操作相对应的非点击操作信息，然后将该非点击操作信息通过约定的通信方式发送至评价设备1；接着，操作信息获取装置111实时地接收该非点击操作信息。In one example, when the evaluation device 1 is a network device, firstly, the user communicates with the browser software or Client software interacts. Taking the mouse as an example, when the user performs a selection operation on the search result page of the browser software of the user device, the user device obtains the user's selection operation on a certain search result item of the search result in real time, for example, the user device first obtains Information related to the operation of the mouse click, such as the position coordinates of the click and the time information of the click, and then when the mouse is released, the information related to the operation of the mouse release, such as the position coordinates of the release and the time information of the release, is obtained and recorded This operation is a selection operation. At the same time, according to the position coordinates under the point and the released position coordinates, determine the search result item corresponding to the selection operation, and determine the corresponding search result item according to the time information under the point and the release time information. The duration of the selection operation; thereby determine the non-click operation information corresponding to the user’s selection operation, and then send the non-click operation information to the evaluation device 1 through the agreed communication method; then, the operation information acquisition device 111 real-time to receive the non-click operation information.
在另一示例中，操作信息获取装置111经由第三方设备提供的应用编程接口(API)，受预定条件或事件触发地、或定期地向该第三方设备发送接收多个用户对待评价的基于相同查询序列的搜索结果执行的非点击操作信息的请求消息，并接收该第三方设备响应于该请求消息返回的多个用户对待评价的基于相同查询序列的搜索结果执行的非点击操作信息。在又一示例中，第三方设备主动向评价设备1推送多个用户对待评价的基于相同查询序列的搜索结果执行的非点击操作信息，操作信息获取装置111接收该多个用户对待评价的基于相同查询序列的搜索结果执行的非点击操作信息。In another example, the operation information acquisition unit 111 sends to the third-party device, through the application programming interface (API) provided by the third-party device, triggered by a predetermined condition or an event, or periodically sends to the third-party device to receive multiple users to be evaluated based on the same A request message for non-click operation information performed on search results of the query sequence, and receiving non-click operation information performed by multiple users to be evaluated based on the same query sequence search results returned by the third-party device in response to the request message. In yet another example, the third-party device actively pushes the non-click operation information performed by multiple users based on the same query sequence to be evaluated to the evaluation device 1, and the operation information acquisition unit 111 receives the information of the multiple users to be evaluated based on the same query sequence. Information about non-click actions performed on search results for the query sequence.
本领域技术人员应能理解上述获取非点击操作信息的方式仅为举例，其他现有的或今后可能出现的获取非点击操作信息的方式如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above method of obtaining non-click operation information is only an example, and other existing or future methods of obtaining non-click operation information, if applicable to the present invention, should also be included in the protection scope of the present invention and is incorporated herein by reference.
接着，评价装置112根据操作信息获取装置111获取的非点击操作信息，对搜索结果进行评价。Next, the evaluation means 112 evaluates the search results according to the non-click operation information acquired by the operation information acquisition means 111 .
在此，根据预定的评价规则对所述搜索结果进行评价的情形包括但不限于以下至少任一项：Here, the situation of evaluating the search result according to predetermined evaluation rules includes but not limited to at least any of the following:
1)用户在获得搜索结果后输入新的查询序列；1) The user inputs a new query sequence after obtaining the search results;
例如，若用户在获得搜索结果后在搜索结果页面中重新输入新的查询序列，则评价装置112确定该搜索结果的评价等级为低；For example, if the user re-inputs a new query sequence in the search result page after obtaining the search result, the evaluation means 112 determines that the evaluation level of the search result is low;
2)用户在搜索结果的各个搜索结果项的划选次数超过预定划选次数阈值；2) The number of times the user selects each search result item in the search results exceeds the predetermined threshold of the number of times of selection;
例如，若用户在搜索结果的各个搜索结果项的划选次数超过预定划选次数阈值范围最大值，则评价装置112确定该搜索结果的评价等级为高；For example, if the number of clicks of each search result item of the search result by the user exceeds the maximum value of the preset threshold range of the number of clicks, the evaluation means 112 determines that the evaluation level of the search result is high;
3)用户在搜索结果的各个搜索结果项的悬停次数超过预定悬停次数阈值；3) The number of times the user hovers over each search result item in the search results exceeds the preset hovering times threshold;
例如，若用户在搜索结果的各个搜索结果项的悬停次数超过预定悬停次数阈值范围最大值，则评价装置112确定该搜索结果的评价等级为高；For example, if the number of times the user hovers over each search result item of the search result exceeds the maximum value of the preset hovering times threshold range, the evaluation means 112 determines that the evaluation level of the search result is high;
4)用户在搜索结果的各个搜索结果项的悬停时长超过预定悬停时长阈值；4) The user's hovering duration of each search result item in the search results exceeds the predetermined hovering duration threshold;
例如，若用户在搜索结果的各个搜索结果项的悬停时长超过预定悬停时长阈值范围最大值，则评价装置112确定该搜索结果的评价等级为高。For example, if the hovering duration of each search result item of the search result exceeds the maximum value of a predetermined hovering duration threshold range, the evaluation means 112 determines that the evaluation level of the search result is high.
本领域技术人员应能理解，上述各项预定的评价规则不仅可以单独用于评价装置对搜索结果进行评价，还可以将其中多项结合用于评价装置对搜索结果进行评价。Those skilled in the art should be able to understand that the above predetermined evaluation rules can not only be used alone for the evaluation device to evaluate the search results, but also can be used in combination for the evaluation device to evaluate the search results.
在此，对所述搜索结果进行评价的评价标准包括但不限于以下任一形式：Here, the evaluation criteria for evaluating the search results include but not limited to any of the following forms:
a)预定评价等级，如预定评价等级为高、中、低；例如，预定多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后均输入新的查询序列且未对各个搜索结果项有划选、悬停等操作，则评价等级为低级；再例如，预定多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后，对该搜索结果的某一搜索结果项的划选次数超过预定划选次数阈值范围最大值，则评价等级为中级；又例如，预设多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后，符合上述对搜索结果进行评价的情形2)、3)、4)中的两项或两项以上，且未输入新的查询序列，则评价等级为高级；a) Predetermined evaluation grades, such as the predetermined evaluation grades are high, medium, and low; for example, more than 80% of the predetermined users all input new query sequences after obtaining search results based on the same query sequence and do not search for each If the result item has operations such as selection and hovering, the evaluation level is low; for another example, after more than 80% of the predetermined users obtain search results based on the same query sequence, a certain search result of the search result If the number of selections of an item exceeds the maximum value of the predetermined number of selection thresholds, the evaluation level is medium; for another example, it is preset that more than 80% of the users who obtain the search results based on the same query sequence meet the above search criteria. If two or more of 2), 3), and 4) are evaluated as a result, and no new query sequence is entered, the evaluation level will be advanced;
b)预定评价标准为好/不好，例如，如果多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后均输入新的查询序列且未对各个搜索结果项有划选、悬停等操作，则评价结果为不好；又例如，如果多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后，符合上述对搜索结果进行评价的情形2)、3)、4)中的两项或两项以上，且为输入新的查询序列，则评价结果为好。b) The predetermined evaluation standard is good/bad, for example, if more than 80% of the multiple users have entered a new query sequence after obtaining search results based on the same query sequence and have not selected each search result item, Hovering and other operations, the evaluation result is not good; for another example, if more than 80% of the multiple users obtain the search results based on the same query sequence, they meet the above conditions 2) and 3) for evaluating the search results , 4) or two or more, and the new query sequence is not input, then the evaluation result is good.
在一示例中，非点击操作信息示出多个用户中85％的用户在基于同一查询序列“丁丁历险记”的搜索结果的搜索结果页面中重新输入新的查询序列，且对该搜索结果的各个搜索结果项没有划选、悬停等操作，则评价装置112对该搜索结果进行评价的评价等级为低。In one example, the non-click operation information shows that 85% of the multiple users re-enter a new query sequence in the search result page based on the search result of the same query sequence "The Adventures of Tintin", and the search result If there is no operation such as selecting or hovering over each search result item, the evaluation device 112 evaluates the search result with a low evaluation level.
优选地，还可根据预定的加权规则给上述预定评价规则赋予权重，评价装置112根据该等权重，加权获取非点击操作信息所对应的评价值以确定评价等级，从而实现对搜索结果进行评价。Preferably, weights can also be assigned to the predetermined evaluation rules according to predetermined weighting rules, and the evaluation device 112 weights the evaluation values corresponding to the non-click operation information to determine the evaluation level according to the weights, so as to realize the evaluation of search results.
在一示例中，预设的加权规则示出：In one example, the preset weighting rules show:
1)当对搜索结果进行评价的情形为：多个用户中80％以上的用户在基于相同查询序列的获得搜索结果后输入新的查询序列，则该情形对应的评价值为-10，且该评价值在用于确定整个非点击操作信息所对应的评价值中所具有的权重为0.5，1) When evaluating the search results: more than 80% of the multiple users input a new query sequence after obtaining the search results based on the same query sequence, then the evaluation value corresponding to this situation is -10, and the The weight of the evaluation value in determining the evaluation value corresponding to the entire non-click operation information is 0.5,
2)当对搜索结果进行评价的情形为：多个用户在搜索结果的各个搜索结果项的划选次数均值超过预定划选次数阈值范围最大值3次，则该情形对应的评价值为10，且该评价值在用于确定整个非点击操作信息所对应的评价值中所具有的权重为0.2，2) When the situation of evaluating the search results is: the average value of the number of selections of each search result item of the search results by multiple users exceeds the maximum value of the preset threshold range of the number of selections for 3 times, then the evaluation value corresponding to this situation is 10, And the weight of the evaluation value in determining the evaluation value corresponding to the entire non-click operation information is 0.2,
3)多个用户在搜索结果的各个搜索结果项的悬停次数均值超过预定悬停次数阈值范围最大值5次，则该情形对应的评价值为10，且该评价值在用于确定整个非点击操作信息所对应的评价值中所具有的权重为0.2，3) The mean value of hovering times of multiple users in each search result item of the search results exceeds the maximum value of the preset hovering times threshold range 5 times, then the evaluation value corresponding to this situation is 10, and the evaluation value is used to determine the entire non- The weight of the evaluation value corresponding to the click operation information is 0.2,
4)多个用户在搜索结果的各个搜索结果项的悬停时长均值超过预定悬停时长阈值范围最大值120秒，则该情形对应的评价值为10，且该评价值在用于确定整个非点击操作信息所对应的评价值中所具有的权重为0.1；4) If the average hovering duration of each search result item of multiple users in the search results exceeds the maximum value of the predetermined hovering duration threshold range of 120 seconds, then the evaluation value corresponding to this situation is 10, and the evaluation value is used to determine the entire non- The weight of the evaluation value corresponding to the click operation information is 0.1;
非点击操作信息示出多个用户在基于同一查询序列“国家大剧院”的搜索结果的搜索结果页面中对各个搜索结果项划选操作次数均值为5次，即超过预定划选次数阈值范围最大值3次，该等用户在该搜索结果页面中对各个搜索结果项的悬停次数均值为6次，即超过预定悬停次数阈值范围最大值5次，该等用户在该搜索结果页面中对各个搜索结果项的悬停时长均超过预定悬停时长阈值范围最大值120秒，且该等用户中90％的用户在该搜索结果页面中未输入新的查询序列，则评价装置112对该等非点击操作信息所对应的评价值加权获取对搜索结果的评价值为5(＝10×0.2+10×0.2+10×0.1)，大于预定权值范围最大值3，确定评价等级为高级。The non-click operation information shows that the average number of selection operations for each search result item by multiple users on the search result page based on the search results of the same query sequence "National Grand Theater" is 5 times, that is, the maximum number of selection operations exceeds the predetermined threshold range value of 3 times, the average number of hovering times of these users on each search result item in the search result page is 6 times, that is, exceeding the maximum value of the preset hovering times threshold range of 5 times, these users hovering over the search result page in the search result page If the hovering duration of each search result item exceeds the maximum value of the preset hovering duration threshold range of 120 seconds, and 90% of these users do not input a new query sequence in the search result page, then the evaluation device 112 evaluates the The evaluation value corresponding to the non-click operation information is obtained by weighting. The evaluation value of the search result is 5 (=10×0.2+10×0.2+10×0.1), which is greater than the maximum value of 3 in the predetermined weight range, and the evaluation level is determined to be advanced.
本领域技术人员应能理解上述对搜索结果进行评价的方式仅为举例，其他现有的或今后可能出现的对搜索结果进行评价的方式如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above method of evaluating search results is only an example, and other existing or future methods of evaluating search results, if applicable to the present invention, should also be included in the protection scope of the present invention and is incorporated herein by reference.
优选地，操作信息获取装置111和评价装置112之间是持续不断地工作。具体地，操作信息获取装置111持续地获取一个或多个用户对待评价的基于相同查询序列的搜索结果的非点击操作信息；接着，评价装置112也持续地根据所述非点击操作信息，对所述搜索结果进行评价。在此，本领域技术人员应理解“持续”是指各装置不断进行上述非点击操作信息的获取、对搜索结果的评价，直至满足预定停止条件，例如请求获取装置111在较长时间内停止对非点击操作信息的获取。Preferably, the operation information acquisition device 111 and the evaluation device 112 work continuously. Specifically, the operation information acquisition means 111 continuously acquires the non-click operation information of one or more users to be evaluated based on the search results of the same query sequence; then, the evaluation means 112 also continuously evaluates all evaluate the search results. Here, those skilled in the art should understand that "continuous" means that each device continuously acquires the above-mentioned non-click operation information and evaluates the search results until a predetermined stop condition is met, such as requesting the acquisition device 111 to stop processing the search results within a relatively long period of time. Acquisition of non-click operation information.
优选地(参照图1)，评价装置112根据非点击操作信息以及该等非点击操作信息发生的时间信息，对搜索结果进行评价。Preferably (refer to FIG. 1 ), the evaluation means 112 evaluates the search results according to the non-click operation information and the time information when the non-click operation information occurs.
在此，所述非点击操作信息包括用户输入新的查询序列的操作信息、对搜索结果的各个搜索结果项的划选操作信息、悬停操作信息以及该等输入、划选、悬停操作发生的时间信息。Here, the non-click operation information includes the operation information of the user inputting a new query sequence, the selection operation information of each search result item in the search result, the hovering operation information, and the occurrence of such input, selection, and hovering operations. time information.
在此，用户的在后操作可以代表对其在前操作的否定，由此在后发生的非点击操作的评价权重大于在前发生的非点击操作的评价权重。Here, the subsequent operation of the user may represent a negation of the user's previous operation, so the evaluation weight of the non-click operation occurring later is greater than the evaluation weight of the non-click operation occurring earlier.
在一示例中，非点击操作信息示出某一用户在获得搜索结果后未输入新的查询序列，同时对搜索结果中的搜索结果项A进行划选操作且该划选操作结束的时间为12:01PM，且该用户对该等搜索结果中的搜索结果项B也进行划选操作且该划选操作结束的时间为12:03PM；则评价装置112根据预定的加权规则，对在后发生的对搜索结果项B的划选操作赋予评价值20，且权重0.8，对搜索结果项A的划选操作赋予评价值10，且的权重0.2，接着对该等非点击操作信息所对应的权重加权获取权值为9(＝(20×0.8+10×0.2)/2)，位于预定权值范围(6至11)中，确定评价等级为中级。In one example, the non-click operation information shows that a certain user did not input a new query sequence after obtaining the search results, and at the same time performed a selection operation on the search result item A in the search results, and the time when the selection operation ended was 12 :01 PM, and the user also performs a selection operation on the search result item B in the search results, and the time when the selection operation ends is 12:03 PM; Assign an evaluation value of 20 to the selection operation of search result item B with a weight of 0.8, and assign an evaluation value of 10 to the selection operation of search result item A with a weight of 0.2, and then weight the corresponding weights of such non-click operation information The obtained weight is 9 (=(20×0.8+10×0.2)/2), which is within the predetermined weight range (6 to 11), and the evaluation level is determined to be medium.
本领域技术人员应能理解上述对搜索结果进行评价的方式仅为举例，其他现有的或今后可能出现的对搜索结果进行评价的方式如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above method of evaluating search results is only an example, and other existing or future methods of evaluating search results, if applicable to the present invention, should also be included in the protection scope of the present invention and is incorporated herein by reference.
更优选地(参照图1)，评价装置112根据非点击操作信息，并结合用户对搜索结果的点击操作信息，对该搜索结果进行评价。More preferably (refer to FIG. 1 ), the evaluating means 112 evaluates the search result according to the non-click operation information combined with the user's click operation information on the search result.
具体地，评价装置112可对非点击操作和点击操作赋予权重，根据预定加权规则，获取加权后的评价值，以确定对搜索结果的评价等级。Specifically, the evaluation means 112 may assign weights to non-click operations and click operations, and obtain weighted evaluation values according to predetermined weighting rules, so as to determine the evaluation level of the search results.
在一示例中，预定加权规则示出：点击操作对应的评价权重为0.6，非点击操作的评价权重为0.4；用户对各个搜索结果项有点击操作，且该点击操作对应的评价值为8，同时根据非点击操作信息确定与其相应的评价等级为高、其权值值为为18，则评价装置112计算获得权值12(＝8×0.6+18×0.4)，12大于预定评价等级阈值范围最大值11，则确定评价等级为高级。In an example, the predetermined weighting rule shows that: the evaluation weight corresponding to the click operation is 0.6, and the evaluation weight corresponding to the non-click operation is 0.4; the user has a click operation on each search result item, and the evaluation value corresponding to the click operation is 8, At the same time, according to the non-click operation information, it is determined that the corresponding evaluation grade is high, and its weight value is 18, then the evaluation device 112 calculates and obtains the weight value 12 (=8×0.6+18×0.4), and 12 is greater than the predetermined evaluation grade threshold value range If the maximum value is 11, it is determined that the evaluation level is advanced.
本领域技术人员应能理解上述对搜索结果进行评价的方式仅为举例，其他现有的或今后可能出现的对搜索结果进行评价的方式如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above method of evaluating search results is only an example, and other existing or future methods of evaluating search results, if applicable to the present invention, should also be included in the protection scope of the present invention and is incorporated herein by reference.
在此，将一个或多个用户对待评价的基于相同查询序列的搜索结果的非点击操作和点击操作相结合，可将用户通过点击搜索结果以满足搜索需求和不用点击即可满足搜索需求的情形均包含至评价标准中，进一步提高对搜索结果进行评价的准确性。Here, combining the non-click operation and click operation of one or more users to be evaluated based on the search results of the same query sequence, the situation that the user can satisfy the search demand by clicking the search result and satisfy the search demand without clicking All are included in the evaluation criteria to further improve the accuracy of evaluation of search results.
图2示出根据本发明一个方面的评价搜索结果的方法流程图。FIG. 2 shows a flowchart of a method of evaluating search results according to one aspect of the present invention.
在此，评价设备1可以是一个网络设备或位于其中的功能模块，其中，网络设备包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云，在此，云由基于云计算(Cloud Computing)的大量计算机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个超级虚拟计算机；评价设备1也可以是用户设备，其包括但不限于任何一种可与用户通过键盘、遥控器、触摸板、或声控设备进行人机交互的电子产品，例如计算机、智能手机、PDA、游戏机、或IPTV等。Here, the evaluation device 1 may be a network device or a functional module located therein, wherein the network device includes but not limited to a computer, a network host, a single network server, a set of multiple network servers or a cloud composed of multiple servers. , the cloud is composed of a large number of computers or network servers based on cloud computing (Cloud Computing), wherein cloud computing is a kind of distributed computing, a super virtual computer composed of a group of loosely coupled computer sets; the evaluation device 1 can also be User equipment, including but not limited to any electronic product that can interact with the user through a keyboard, remote control, touch pad, or voice-activated device, such as a computer, smart phone, PDA, game console, or IPTV.
以下参照图2来对评价设备1评价搜索结果的过程进行详细描述：The process of evaluating the search results by the evaluation device 1 is described in detail below with reference to FIG. 2 :
具体地，在步骤S1中，评价设备1获取一个或多个用户对待评价的基于相同查询序列的搜索结果的非点击操作信息。Specifically, in step S1, the evaluation device 1 acquires non-click operation information of one or more search results based on the same query sequence to be evaluated by one or more users.
在此，所述非点击操作信息包括但不限于以下任一项：Here, the non-click operation information includes but not limited to any of the following:
1)用户输入新的查询序列的操作信息；例如，在已获得搜索结果后，用户在搜索结果页面中的搜索输入框又进行输入操作，且该输入操作对应的新的查询序列与前次查询序列相关，例如两个查询序列中有部分查询关键字完全相同，则可确定用户在其搜索需求未被满足的情况下输入新的查询序列；1) The operation information of the user inputting a new query sequence; for example, after obtaining the search results, the user performs another input operation in the search input box on the search result page, and the new query sequence corresponding to the input operation is the same as the previous query Sequence correlation, for example, if some of the query keywords in two query sequences are identical, it can be determined that the user enters a new query sequence when his search needs are not met;
2)用户对搜索结果的各个搜索结果项的划选操作信息；其中，所述划选操作信息包括但不限于用户对搜索结果的各个搜索结果项进行划选操作的次数；例如，根据鼠标点下的操作相关信息，如点下的位置坐标及点下的时间信息，以及相应地释放鼠标时该鼠标释放的操作相关信息，如释放的位置坐标及释放的时间信息，以记录该等操作为一次划选操作，同时，根据该点下的位置坐标与释放的位置坐标，确定该次划选操作对应的搜索结果项，并根据该点下的时间信息与放开的时间信息，确定该次划选操作的时长；2) The user's selection operation information on each search result item of the search result; wherein, the selection operation information includes but not limited to the number of times the user performs a selection operation on each search result item of the search result; for example, according to the mouse click Information related to the operation under the click, such as the position coordinates of the click and the time information of the click, and the operation related information of the mouse when the mouse is released correspondingly, such as the position coordinates of the release and the time information of the release, so as to record such operations as A swipe operation, at the same time, according to the position coordinates under the point and the released position coordinates, determine the search result item corresponding to the swipe operation, and determine the time information according to the time information under the point and the release time information. The duration of the selection operation;
3)用户对搜索结果的各个搜索结果项的悬停操作信息；3) The hover operation information of each search result item of the search result by the user;
其中，所述用户对所述搜索结果的各个搜索结果项的悬停操作信息包括以下至少任一项：Wherein, the user's hover operation information on each search result item of the search result includes at least any of the following:
a)用户在搜索结果的各个搜索结果项上的悬停操作次数；例如，实时地获取鼠标移入搜索结果的某个搜索结果项所在区域的时间以及移出该区域的时间，当移入时间与移出时间的时间间隔超过预定时间间隔阈值时，记录为一次悬停操作，进而统计用户在该搜索结果所在搜索结果页面中的总的悬停操作次数；a) The number of times the user hovers over each search result item in the search results; for example, the time when the mouse moves into the area where a certain search result item of the search result is located and the time when the mouse moves out of the area are obtained in real time. When the time interval exceeds the predetermined time interval threshold, it is recorded as a hovering operation, and then the total number of hovering operations of the user on the search result page where the search result is located is counted;
b)用户在搜索结果的各个搜索结果项上的每次悬停操作的持续时间；例如，实时地获取鼠标移入搜索结果的某个搜索结果项所在区域的时间以及移出该区域的时间，记录移入时间与移出时间的时间间隔，即每次悬停操作的持续时间。b) The duration of each hovering operation of the user on each search result item of the search result; for example, obtain the time when the mouse moves into the area where a certain search result item of the search result is located and the time when the mouse moves out of the area in real time, and record the time when the mouse moves in The interval between time and moveout time, i.e. the duration of each hover action.
本领域技术人员应能理解上述悬停操作信息仅为举例，其他现有的或今后可能出现的悬停操作信息如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above-mentioned hovering operation information is only an example, and other existing or future hovering operation information, if applicable to the present invention, should also be included in the protection scope of the present invention, and shall be incorporated by reference included here.
本领域技术人员应能理解上述非点击操作信息仅为举例，其他现有的或今后可能出现的非点击操作信息如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above-mentioned non-click operation information is only an example, and other existing or future non-click operation information, if applicable to the present invention, should also be included in the protection scope of the present invention, and shall be cited included here.
在此，所述获取非点击操作信息的方式包括但不限于以下任一情形：Here, the methods of obtaining non-click operation information include but are not limited to any of the following situations:
1)在步骤S1中，评价设备1获取一个用户通过用户设备对待评价的基于相同查询序列的搜索结果执行的非点击操作信息；1) In step S1, the evaluation device 1 obtains the non-click operation information performed by a user through the user device on the search results based on the same query sequence to be evaluated;
2)在步骤S1中，评价设备1应事件触发实时地、或周期地接收来自第三方设备发送的多个用户对待评价的基于相同查询序列的搜索结果的非点击操作信息。2) In step S1, the evaluation device 1 receives the non-click operation information of the search results based on the same query sequence to be evaluated by multiple users sent from the third-party device in real time or periodically in response to an event trigger.
在一示例中，当评价设备1为网络设备时，首先，用户借助与用户设备的交互方式，包括但不限于键盘、鼠标、遥控器、触摸板、或手写设备，与用户设备的浏览器软件或客户端软件进行交互。以鼠标为例，用户在用户设备的浏览器软件的搜索结果页面中进行划选操作时，该用户设备实时地获取用户对搜索结果的某一搜索结果项的划选操作，例如用户设备首先获取鼠标点下的操作相关信息，如点下的位置坐标及点下的时间信息，接着当释放该鼠标时，获取该鼠标释放的操作相关信息，如释放的位置坐标及释放的时间信息，并记录该次操作为划选操作，同时，根据该点下的位置坐标与释放的位置坐标，确定该次划选操作对应的搜索结果项，并根据该点下的时间信息与释放的时间信息，确定该次划选操作的时长；从而确定与该用户划选操作相对应的非点击操作信息，然后将该非点击操作信息通过约定的通信方式发送至评价设备1；接着，在步骤S1中，评价设备1实时地接收该非点击操作信息。In one example, when the evaluation device 1 is a network device, firstly, the user interacts with the user device through the interaction method, including but not limited to keyboard, mouse, remote control, touch pad, or handwriting device, and the browser software of the user device Or client software to interact. Taking the mouse as an example, when the user performs a selection operation on the search result page of the browser software of the user device, the user device obtains the user's selection operation on a certain search result item of the search result in real time, for example, the user device first obtains Information related to the operation of the mouse click, such as the position coordinates of the click and the time information of the click, and then when the mouse is released, the information related to the operation of the mouse release, such as the position coordinates of the release and the time information of the release, is obtained and recorded This operation is a selection operation. At the same time, according to the position coordinates under the point and the released position coordinates, determine the search result item corresponding to the selection operation, and determine the corresponding search result item according to the time information under the point and the release time information. The duration of the selection operation; thereby determine the non-click operation information corresponding to the user’s selection operation, and then send the non-click operation information to the evaluation device 1 through the agreed communication method; then, in step S1, evaluate Device 1 receives the non-click operation information in real time.
在另一示例中，在步骤S1中，评价设备1经由第三方设备提供的应用编程接口(API)，受预定条件或事件触发地、或定期地向该第三方设备发送接收多个用户对待评价的基于相同查询序列的搜索结果执行的非点击操作信息的请求消息，并接收该第三方设备响应于该请求消息返回的多个用户对待评价的基于相同查询序列的搜索结果执行的非点击操作信息。在又一示例中，第三方设备主动向评价设备1推送多个用户对待评价的基于相同查询序列的搜索结果执行的非点击操作信息，在步骤S1中，评价设备1接收该多个用户对待评价的基于相同查询序列的搜索结果执行的非点击操作信息。In another example, in step S1, the evaluation device 1 is triggered by predetermined conditions or events, or periodically sends to the third-party device via an application programming interface (API) provided by the third-party device to receive multiple user evaluations. A request message for the non-click operation information performed based on the search results of the same query sequence, and receiving the non-click operation information performed by multiple users to be evaluated based on the same query sequence search results returned by the third-party device in response to the request message . In yet another example, the third-party device actively pushes to the evaluation device 1 non-click operation information performed by multiple users based on the search results of the same query sequence to be evaluated, and in step S1, the evaluation device 1 receives the Information about non-click actions performed on search results based on the same query sequence.
本领域技术人员应能理解上述获取非点击操作信息的方式仅为举例，其他现有的或今后可能出现的获取非点击操作信息的方式如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above method of obtaining non-click operation information is only an example, and other existing or future methods of obtaining non-click operation information, if applicable to the present invention, should also be included in the protection scope of the present invention and is incorporated herein by reference.
接着，在步骤S2中，评价设备1根据其在步骤S1中获取的非点击操作信息，对搜索结果进行评价。Next, in step S2, the evaluation device 1 evaluates the search results according to the non-click operation information it acquires in step S1.
在此，根据预定的评价规则对所述搜索结果进行评价的情形包括但不限于以下至少任一项：Here, the situation of evaluating the search result according to predetermined evaluation rules includes but not limited to at least any of the following:
1)用户在获得搜索结果后输入新的查询序列；1) The user inputs a new query sequence after obtaining the search results;
例如，若用户在获得搜索结果后在搜索结果页面中重新输入新的查询序列，则在步骤S2中，评价设备1确定该搜索结果的评价等级为低；For example, if the user re-inputs a new query sequence in the search result page after obtaining the search result, then in step S2, the evaluation device 1 determines that the evaluation level of the search result is low;
2)用户在搜索结果的各个搜索结果项的划选次数超过预定划选次数阈值；2) The number of times the user selects each search result item in the search results exceeds the predetermined threshold of the number of times of selection;
例如，若用户在搜索结果的各个搜索结果项的划选次数超过预定划选次数阈值范围最大值，则在步骤S2中，评价设备1确定该搜索结果的评价等级为高；For example, if the user's number of ticks in each search result item of the search result exceeds the maximum value of the predetermined threshold range of the number of ticks, then in step S2, the evaluation device 1 determines that the evaluation level of the search result is high;
3)用户在搜索结果的各个搜索结果项的悬停次数超过预定悬停次数阈值；3) The number of times the user hovers over each search result item in the search results exceeds the preset hovering times threshold;
例如，若用户在搜索结果的各个搜索结果项的悬停次数超过预定悬停次数阈值范围最大值，则在步骤S2中，评价设备1确定该搜索结果的评价等级为高；For example, if the number of times the user hovers over each search result item of the search result exceeds the maximum value of the preset hovering times threshold range, then in step S2, the evaluation device 1 determines that the evaluation level of the search result is high;
4)用户在搜索结果的各个搜索结果项的悬停时长超过预定悬停时长阈值；4) The user's hovering duration of each search result item in the search results exceeds the predetermined hovering duration threshold;
例如，若用户在搜索结果的各个搜索结果项的悬停时长超过预定悬停时长阈值范围最大值，则在步骤S2中，评价设备1确定该搜索结果的评价等级为高。For example, if the user's hovering duration of each search result item in the search result exceeds the maximum value of a predetermined hovering duration threshold range, then in step S2, the evaluation device 1 determines that the evaluation level of the search result is high.
本领域技术人员应能理解，上述各项预定的评价规则不仅可以单独用于评价设备1对搜索结果进行评价，还可以将其中多项结合用于评价设备1对搜索结果进行评价。Those skilled in the art should understand that the above predetermined evaluation rules can not only be used alone for the evaluation device 1 to evaluate the search results, but also can be used in combination for the evaluation device 1 to evaluate the search results.
在此，对所述搜索结果进行评价的评价标准包括但不限于以下任一形式：Here, the evaluation criteria for evaluating the search results include but not limited to any of the following forms:
a)预定评价等级，如预定评价等级为高、中、低；例如，预定多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后均输入新的查询序列且未对各个搜索结果项有划选、悬停等操作，则评价等级为低级；再例如，预定多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后，对该搜索结果的某一搜索结果项的划选次数超过预定划选次数阈值范围最大值，则评价等级为中级；又例如，预设多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后，符合上述对搜索结果进行评价的情形2)、3)、4)中的两项或两项以上，且未输入新的查询序列，则评价等级为高级；a) Predetermined evaluation grades, such as the predetermined evaluation grades are high, medium, and low; for example, more than 80% of the predetermined users all input new query sequences after obtaining search results based on the same query sequence and do not search for each If the result item has operations such as selection and hovering, the evaluation level is low; for another example, after more than 80% of the predetermined users obtain search results based on the same query sequence, a certain search result of the search result If the number of selections of an item exceeds the maximum value of the predetermined number of selection thresholds, the evaluation level is medium; for another example, it is preset that more than 80% of the users who obtain the search results based on the same query sequence meet the above search criteria. If two or more of 2), 3), and 4) are evaluated as a result, and no new query sequence is entered, the evaluation level will be advanced;
b)预定评价标准为好/不好，例如，如果多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后均输入新的查询序列且未对各个搜索结果项有划选、悬停等操作，则评价结果为不好；又例如，如果多个用户中80％以上的用户在获得基于相同查询序列的搜索结果后，符合上述对搜索结果进行评价的情形2)、3)、4)中的两项或两项以上，且为输入新的查询序列，则评价结果为好。b) The predetermined evaluation standard is good/bad, for example, if more than 80% of the multiple users have entered a new query sequence after obtaining search results based on the same query sequence and have not selected each search result item, Hovering and other operations, the evaluation result is not good; for another example, if more than 80% of the multiple users obtain the search results based on the same query sequence, they meet the above conditions 2) and 3) for evaluating the search results , 4) or two or more, and the new query sequence is not input, then the evaluation result is good.
在一示例中，非点击操作信息示出多个用户中85％的用户在基于同一查询序列“丁丁历险记”的搜索结果的搜索结果页面中重新输入新的查询序列，且对该搜索结果的各个搜索结果项没有划选、悬停等操作，则在步骤S2中，评价设备1对该搜索结果进行评价的评价等级为低。In one example, the non-click operation information shows that 85% of the multiple users re-enter a new query sequence in the search result page based on the search result of the same query sequence "The Adventures of Tintin", and the search result If there is no operation such as selecting or hovering over each search result item, then in step S2, the evaluation level of the search result evaluated by the evaluation device 1 is low.
优选地，还可根据预定的加权规则给上述预定评价规则赋予权重，在步骤S2中，评价设备1根据该等权重，加权获取非点击操作信息所对应的评价值以确定评价等级，从而实现对搜索结果进行评价。Preferably, weights can also be assigned to the above-mentioned predetermined evaluation rules according to predetermined weighting rules. In step S2, the evaluation device 1 weights the evaluation values corresponding to the non-click operation information to determine the evaluation level according to the weights, so as to realize the evaluation level. Review the search results.
在一示例中，预设的加权规则示出：In one example, the preset weighting rules show:
1)当对搜索结果进行评价的情形为：多个用户中80％以上的用户在基于相同查询序列的获得搜索结果后输入新的查询序列，则该情形对应的评价值为-10，且该评价值在用于确定整个非点击操作信息所对应的评价值中所具有的权重为0.5，1) When evaluating the search results: more than 80% of the multiple users input a new query sequence after obtaining the search results based on the same query sequence, then the evaluation value corresponding to this situation is -10, and the The weight of the evaluation value in determining the evaluation value corresponding to the entire non-click operation information is 0.5,
2)当对搜索结果进行评价的情形为：多个用户在搜索结果的各个搜索结果项的划选次数均值超过预定划选次数阈值范围最大值3次，则该情形对应的评价值为10，且该评价值在用于确定整个非点击操作信息所对应的评价值中所具有的权重为0.2，2) When the situation of evaluating the search results is: the average value of the number of selections of each search result item of the search results by multiple users exceeds the maximum value of the preset threshold range of the number of selections for 3 times, then the evaluation value corresponding to this situation is 10, And the weight of the evaluation value in determining the evaluation value corresponding to the entire non-click operation information is 0.2,
3)多个用户在搜索结果的各个搜索结果项的悬停次数均值超过预定悬停次数阈值范围最大值5次，则该情形对应的评价值为10，且该评价值在用于确定整个非点击操作信息所对应的评价值中所具有的权重为0.2，3) The mean value of hovering times of multiple users in each search result item of the search results exceeds the maximum value of the preset hovering times threshold range 5 times, then the evaluation value corresponding to this situation is 10, and the evaluation value is used to determine the entire non- The weight of the evaluation value corresponding to the click operation information is 0.2,
4)多个用户在搜索结果的各个搜索结果项的悬停时长均值超过预定悬停时长阈值范围最大值120秒，则该情形对应的评价值为10，且该评价值在用于确定整个非点击操作信息所对应的评价值中所具有的权重为0.1；4) If the average hovering duration of each search result item of multiple users in the search results exceeds the maximum value of the predetermined hovering duration threshold range of 120 seconds, then the evaluation value corresponding to this situation is 10, and the evaluation value is used to determine the entire non- The weight of the evaluation value corresponding to the click operation information is 0.1;
非点击操作信息示出多个用户在基于同一查询序列“国家大剧院”的搜索结果的搜索结果页面中对各个搜索结果项划选操作次数均值为5次，即超过预定划选次数阈值范围最大值3次，该等用户在该搜索结果页面中对各个搜索结果项的悬停次数均值为6次，即超过预定悬停次数阈值范围最大值5次，该等用户在该搜索结果页面中对各个搜索结果项的悬停时长均超过预定悬停时长阈值范围最大值120秒，且该等用户中90％的用户在该搜索结果页面中未输入新的查询序列，则在步骤S2中，评价设备1对该等非点击操作信息所对应的评价值加权获取对搜索结果的评价值为5(＝10×0.2+10×0.2+10×0.1)，大于预定权值范围最大值3，确定评价等级为高级。The non-click operation information shows that the average number of selection operations for each search result item by multiple users on the search result page based on the search results of the same query sequence "National Grand Theater" is 5 times, that is, the maximum number of selection operations exceeds the predetermined threshold range value of 3 times, the average number of hovering times of these users on each search result item in the search result page is 6 times, that is, exceeding the maximum value of the preset hovering times threshold range of 5 times, these users hovering over the search result page in the search result page If the hover duration of each search result item exceeds the maximum value of the preset hover duration threshold range of 120 seconds, and 90% of these users have not input a new query sequence on the search result page, then in step S2, evaluate Device 1 weights the evaluation value corresponding to such non-click operation information to obtain an evaluation value of 5 (=10×0.2+10×0.2+10×0.1) for the search result, which is greater than the maximum value of 3 in the predetermined weight range, and determines the evaluation The grade is advanced.
本领域技术人员应能理解上述对搜索结果进行评价的方式仅为举例，其他现有的或今后可能出现的对搜索结果进行评价的方式如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above method of evaluating search results is only an example, and other existing or future methods of evaluating search results, if applicable to the present invention, should also be included in the protection scope of the present invention and is incorporated herein by reference.
优选地，评价设备1在步骤S1和步骤S2中是持续不断地工作。具体地，在步骤S1中，评价设备1持续地获取一个或多个用户对待评价的基于相同查询序列的搜索结果的非点击操作信息；接着，在步骤S2中，评价设备1也持续地根据所述非点击操作信息，对所述搜索结果进行评价。在此，本领域技术人员应理解“持续”是指评价设备1在各步骤中不断进行上述非点击操作信息的获取、对搜索结果的评价，直至满足预定停止条件，例如评价设备1在较长时间内停止对非点击操作信息的获取。Preferably, the evaluation device 1 works continuously in steps S1 and S2. Specifically, in step S1, the evaluation device 1 continuously obtains the non-click operation information of one or more users to be evaluated based on the search results of the same query sequence; then, in step S2, the evaluation device 1 also continuously obtains the The above non-click operation information is used to evaluate the search results. Here, those skilled in the art should understand that "continuous" means that the evaluation device 1 continuously acquires the above-mentioned non-click operation information and evaluates the search results in each step until a predetermined stop condition is met, for example, the evaluation device 1 is in a long period of time. Stop the acquisition of non-click operation information within a certain time.
优选地(参照图2)，在步骤S2中，评价设备1根据非点击操作信息以及该等非点击操作信息发生的时间信息，对搜索结果进行评价。Preferably (refer to FIG. 2 ), in step S2, the evaluation device 1 evaluates the search results according to the non-click operation information and the time information when the non-click operation information occurs.
在此，所述非点击操作信息包括用户输入新的查询序列的操作信息、对搜索结果的各个搜索结果项的划选操作信息、悬停操作信息以及该等输入、划选、悬停操作发生的时间信息。Here, the non-click operation information includes the operation information of the user inputting a new query sequence, the selection operation information of each search result item in the search result, the hovering operation information, and the occurrence of such input, selection, and hovering operations. time information.
在此，用户的在后操作可以代表对其在前操作的否定，由此在后发生的非点击操作的评价权重大于在前发生的非点击操作的评价权重。Here, the subsequent operation of the user may represent a negation of the user's previous operation, so the evaluation weight of the non-click operation occurring later is greater than the evaluation weight of the non-click operation occurring earlier.
在一示例中，非点击操作信息示出某一用户在获得搜索结果后未输入新的查询序列，同时对搜索结果中的搜索结果项A进行划选操作且该划选操作结束的时间为12:01PM，且该用户对该等搜索结果中的搜索结果项B也进行划选操作且该划选操作结束的时间为12:03PM；则在步骤S2中，评价设备1根据预定的加权规则，对在后发生的对搜索结果项B的划选操作赋予评价值20，且权重0.8，对搜索结果项A的划选操作赋予评价值10，且的权重0.2，接着对该等非点击操作信息所对应的权重加权获取权值为9(＝(20×0.8+10×0.2)/2)，位于预定权值范围(6至11)中，确定评价等级为中级。In one example, the non-click operation information shows that a certain user did not input a new query sequence after obtaining the search results, and at the same time performed a selection operation on the search result item A in the search results, and the time when the selection operation ended was 12 :01PM, and the user also performs a selection operation on the search result item B in the search results and the time when the selection operation ends is 12:03PM; then in step S2, the evaluation device 1 according to the predetermined weighting rule, Assign an evaluation value of 20 to the subsequent selection operation on search result item B with a weight of 0.8, and assign an evaluation value of 10 to the selection operation on search result item A with a weight of 0.2, and then assign such non-click operation information The corresponding weighted acquisition weight is 9 (=(20×0.8+10×0.2)/2), which is within the predetermined weight range (6 to 11), and the evaluation level is determined to be medium.
本领域技术人员应能理解上述对搜索结果进行评价的方式仅为举例，其他现有的或今后可能出现的对搜索结果进行评价的方式如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above method of evaluating search results is only an example, and other existing or future methods of evaluating search results, if applicable to the present invention, should also be included in the protection scope of the present invention and is incorporated herein by reference.
更优选地(参照图2)，在步骤S2中，评价设备1根据非点击操作信息，并结合用户对搜索结果的点击操作信息，对该搜索结果进行评价。More preferably (refer to FIG. 2 ), in step S2, the evaluation device 1 evaluates the search result according to the non-click operation information combined with the user's click operation information on the search result.
具体地，在步骤S2中，评价设备1可对非点击操作和点击操作赋予权重，根据预定加权规则，获取加权后的评价值，以确定对搜索结果的评价等级。Specifically, in step S2, the evaluation device 1 may assign weights to non-click operations and click operations, and obtain weighted evaluation values according to predetermined weighting rules to determine the evaluation level of the search results.
在一示例中，预定加权规则示出：点击操作对应的评价权重为0.6，非点击操作的评价权重为0.4；用户对各个搜索结果项有点击操作，且该点击操作对应的评价值为8，同时根据非点击操作信息确定与其相应的评价等级为高、其权值值为为18，则在步骤S2中，评价设备1计算获得权值12(＝8×0.6+18×0.4)，12大于预定评价等级阈值范围最大值11，则确定评价等级为高级。In an example, the predetermined weighting rule shows that: the evaluation weight corresponding to the click operation is 0.6, and the evaluation weight corresponding to the non-click operation is 0.4; the user has a click operation on each search result item, and the evaluation value corresponding to the click operation is 8, At the same time, according to the non-click operation information, it is determined that the corresponding evaluation level is high, and its weight value is 18, then in step S2, the evaluation device 1 calculates and obtains a weight value of 12 (=8×0.6+18×0.4), and 12 is greater than If the maximum value of the predetermined evaluation level threshold range is 11, then the evaluation level is determined to be advanced.
本领域技术人员应能理解上述对搜索结果进行评价的方式仅为举例，其他现有的或今后可能出现的对搜索结果进行评价的方式如可适用于本发明，也应包含在本发明保护范围以内，并以引用方式包含于此。Those skilled in the art should be able to understand that the above method of evaluating search results is only an example, and other existing or future methods of evaluating search results, if applicable to the present invention, should also be included in the protection scope of the present invention and is incorporated herein by reference.
在此，将一个或多个用户对待评价的基于相同查询序列的搜索结果的非点击操作和点击操作相结合，可将用户通过点击搜索结果以满足搜索需求和不用点击即可满足搜索需求的情形均包含至评价标准中，进一步提高对搜索结果进行评价的准确性。Here, combining the non-click operation and click operation of one or more users to be evaluated based on the search results of the same query sequence, the situation that the user can satisfy the search demand by clicking the search result and satisfy the search demand without clicking All are included in the evaluation criteria to further improve the accuracy of evaluation of search results.
对于本领域技术人员而言，显然本发明不限于上述示范性实施例的细节，而且在不背离本发明的精神或基本特征的情况下，能够以其他的具体形式实现本发明。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本发明的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本发明内。不应将权利要求中的任何附图标记视为限制所涉及的权利要求。此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。装置权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第一，第二等词语用来表示名称，而并不表示任何特定的顺序。It will be apparent to those skilled in the art that the invention is not limited to the details of the above-described exemplary embodiments, but that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Accordingly, the embodiments should be regarded in all points of view as exemplary and not restrictive, the scope of the invention being defined by the appended claims rather than the foregoing description, and it is therefore intended that the scope of the invention be defined by the appended claims rather than by the foregoing description. All changes within the meaning and range of equivalents of the elements are embraced in the present invention. Any reference sign in a claim should not be construed as limiting the claim concerned. In addition, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means stated in the device claims may also be realized by one unit or device through software or hardware. The words first, second, etc. are used to denote names and do not imply any particular order.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110457424.7ACN103186619B (en) | 2011-12-30 | 2011-12-30 | A kind of method and apparatus of the evaluation search result based on non click operation information |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110457424.7ACN103186619B (en) | 2011-12-30 | 2011-12-30 | A kind of method and apparatus of the evaluation search result based on non click operation information |
| Publication Number | Publication Date |
|---|---|
| CN103186619A CN103186619A (en) | 2013-07-03 |
| CN103186619Btrue CN103186619B (en) | 2018-08-07 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110457424.7AActiveCN103186619B (en) | 2011-12-30 | 2011-12-30 | A kind of method and apparatus of the evaluation search result based on non click operation information |
| Country | Link |
|---|---|
| CN (1) | CN103186619B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9626439B2 (en)* | 2013-05-31 | 2017-04-18 | Xilopix | Method for searching in a database |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6269361B1 (en)* | 1999-05-28 | 2001-07-31 | Goto.Com | System and method for influencing a position on a search result list generated by a computer network search engine |
| CN1818908A (en)* | 2006-03-16 | 2006-08-16 | 董崇军 | Feedbakc information use of searcher in search engine |
| CN101030210A (en)* | 2006-10-08 | 2007-09-05 | 胡继强 | Method for searching sort by user action |
| CN101454780A (en)* | 2006-03-30 | 2009-06-10 | 谷歌公司 | Method for Generating Website Profile Based on Monitoring User Behavior |
| CN102298615A (en)* | 2011-07-29 | 2011-12-28 | 百度在线网络技术（北京）有限公司 | Method for displaying research result realized by computer and equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6269361B1 (en)* | 1999-05-28 | 2001-07-31 | Goto.Com | System and method for influencing a position on a search result list generated by a computer network search engine |
| CN1818908A (en)* | 2006-03-16 | 2006-08-16 | 董崇军 | Feedbakc information use of searcher in search engine |
| CN101454780A (en)* | 2006-03-30 | 2009-06-10 | 谷歌公司 | Method for Generating Website Profile Based on Monitoring User Behavior |
| CN101030210A (en)* | 2006-10-08 | 2007-09-05 | 胡继强 | Method for searching sort by user action |
| CN102298615A (en)* | 2011-07-29 | 2011-12-28 | 百度在线网络技术（北京）有限公司 | Method for displaying research result realized by computer and equipment |
| Publication number | Publication date |
|---|---|
| CN103186619A (en) | 2013-07-03 |
| Publication | Publication Date | Title |
|---|---|---|
| US20210329094A1 (en) | Discovering signature of electronic social networks | |
| CN109684118B (en) | Abnormal data detection method, apparatus, device, and computer-readable storage medium | |
| CN106489146B (en) | Query rewriting using session information | |
| JP4550781B2 (en) | Real-time soaring search word detection method and real-time soaring search word detection system | |
| US11593906B2 (en) | Image recognition based content item selection | |
| US8910188B1 (en) | Deterministic data processing | |
| JP5438087B2 (en) | Advertisement distribution device | |
| EP2498212A1 (en) | Information processing apparatus, terminal apparatus, information presentation system, calculation method of evaluation scores, and program. | |
| CN105247507A (en) | Brand Impact Score | |
| US9245035B2 (en) | Information processing system, information processing method, program, and non-transitory information storage medium | |
| CN105701097A (en) | Social-network-platform-based public opinion analysis method and system | |
| CN101957845A (en) | An online application system and its implementation method | |
| CN102915481B (en) | A kind of method, device and equipment for user account is managed | |
| CN111310044B (en) | Method, device, equipment and storage medium for extracting page element information | |
| Speicher et al. | TellMyRelevance! predicting the relevance of web search results from cursor interactions | |
| CN116521511A (en) | Risk code pre-detection method, device, equipment and storage medium | |
| US9880879B1 (en) | Identifying task instance outliers based on metric data in a large scale parallel processing system | |
| CN103186604A (en) | Method, device and equipment for determining satisfaction degree of user on search result | |
| CN109034867A (en) | click traffic detection method, device and storage medium | |
| US20120005021A1 (en) | Selecting advertisements using user search history segmentation | |
| US20140365571A1 (en) | Automatically Determining Veracity of Documents Posted in a Public Forum | |
| CN104025086A (en) | Triggering social pages | |
| US8825698B1 (en) | Showing prominent users for information retrieval requests | |
| CN103186619B (en) | A kind of method and apparatus of the evaluation search result based on non click operation information | |
| CN107507023B (en) | Information delivery method and device |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |