技术领域Technical Field
本发明属于数据存储技术领域，尤其涉及一种信息存储方法、装置、服务器及介质。The present invention belongs to the technical field of data storage, and in particular, relates to an information storage method, device, server and medium.
背景技术Background technique
随着电子设备的普及，许多应用场景下，均需要每隔一段时间即获取电子设备的设备信息，来对电子设备或电子设备上登录的用户账号进行分析、统计以及管理等，例如推荐系统中则需要根据设备信息来周期性的更新该电子设备对应的推荐内容，购物系统中需要根据设备信息来判断当前申请购物券的账号是否为重复申请等。With the popularization of electronic devices, in many application scenarios, it is necessary to obtain the device information of electronic devices at regular intervals to analyze, count and manage the electronic devices or the user accounts logged in to the electronic devices. For example, in the recommendation system, it is necessary to periodically update the recommended content corresponding to the electronic device based on the device information. In the shopping system, it is necessary to determine whether the account currently applying for shopping vouchers is a duplicate application based on the device information.
在上述过程中，所获取的设备信息由于对实时性具有一定的要求，因此，设备信息会首先存储在Redis缓存中，之后再根据缓存中的设备信息进行后续的处理。In the above process, the acquired device information has certain requirements for real-time performance. Therefore, the device information will be first stored in the Redis cache, and then subsequent processing will be performed based on the device information in the cache.
这种情况下，由于一个缓存需要存储大量电子设备的设备信息，使缓存具有很大的存储压力，且容易出现缓存空间不够的情况。In this case, since a cache needs to store device information of a large number of electronic devices, the cache is under great storage pressure and is prone to insufficient cache space.
因此，如何提供一种降低存储压力的信息存储方法是业界所要解决的问题。Therefore, how to provide an information storage method that reduces storage pressure is a problem that the industry needs to solve.
发明内容Summary of the invention
本发明实施例提供了一种信息存储方法、装置、服务器及介质，能够降低缓存在存储设备信息时的存储压力。Embodiments of the present invention provide an information storage method, apparatus, server, and medium, which can reduce storage pressure when caching information in a storage device.
第一方面，本发明实施例提供一种信息存储方法，包括：In a first aspect, an embodiment of the present invention provides an information storage method, comprising:
获取第i次采集到的第一设备的第一设备信息；1≤i≤N，N为缓存中保存的每个多维有序向量对应的预设采集次数；i和N均为正整数；Obtaining the first device information of the first device collected for the i-th time; 1≤i≤N, N is the preset number of collection times corresponding to each multidimensional ordered vector stored in the cache; i and N are both positive integers;
将所述第一设备信息与所述多维有序向量中的基准信息进行比较，提取所述第一设备信息中与所述基准信息不同的目标关键值信息；其中，所述多维有序向量与所述第一设备对应；Compare the first device information with the reference information in the multidimensional ordered vector, and extract target key value information in the first device information that is different from the reference information; wherein the multidimensional ordered vector corresponds to the first device;
将所述目标关键值信息保存至所述多维有序向量中所述目标关键值信息对应的目标维度中。The target key value information is saved in the target dimension corresponding to the target key value information in the multidimensional ordered vector.
第二方面，本发明实施例提供了一种信息存储装置，包括：In a second aspect, an embodiment of the present invention provides an information storage device, including:
信息获取模块，用于获取第i次采集到的第一设备的第一设备信息；1≤i≤N，N为缓存中保存的每个多维有序向量对应的预设采集次数；i和N均为正整数；An information acquisition module, used to acquire the first device information of the first device acquired for the i-th time; 1≤i≤N, N is the preset number of acquisitions corresponding to each multidimensional ordered vector stored in the cache; i and N are both positive integers;
信息提取模块，用于将所述第一设备信息与多维有序向量中的基准信息进行比较，提取所述第一设备信息中与所述基准信息不同的目标关键值信息；其中，所述多维有序向量与所述第一设备对应；an information extraction module, configured to compare the first device information with the reference information in the multidimensional ordered vector, and extract target key value information in the first device information that is different from the reference information; wherein the multidimensional ordered vector corresponds to the first device;
第一保存模块，用于将所述目标关键值信息保存至所述多维有序向量中所述目标关键值信息对应的目标维度中。The first saving module is used to save the target key value information into the target dimension corresponding to the target key value information in the multi-dimensional ordered vector.
第三方面，本发明实施例提供一种服务器，所述服务器包括：处理器以及存储有计算机程序指令的存储器；In a third aspect, an embodiment of the present invention provides a server, the server comprising: a processor and a memory storing computer program instructions;
所述处理器执行所述计算机程序指令时实现如第一方面所述的信息存储方法。When the processor executes the computer program instructions, the information storage method as described in the first aspect is implemented.
第四方面，本发明实施例提供一种计算机存储介质，所述计算机存储介质上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现如第一方面所述的信息存储方法。In a fourth aspect, an embodiment of the present invention provides a computer storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the information storage method as described in the first aspect is implemented.
本发明实施例的方法，在获取第i次采集到的第一设备的第一设备信息后，会将第一设备信息与多维有序向量中设置的基准信息进行比较，之后基于比较结果，提取出第一设备信息中与基准信息不同的目标关键字信息，并将目标关键字信息保存至多维有序向量的目标维度中，该多维有序向量保存于后台服务器对应的缓存中。这种情况下，使得多维有序向量中，不会重复存储与基准信息相同的信息，从而减少了缓存中的重复信息，减少了设备信息对缓存空间的占用，从而降低了缓存的存储压力，提高缓存中的存储效率。The method of the embodiment of the present invention, after obtaining the first device information of the first device collected for the i-th time, will compare the first device information with the reference information set in the multidimensional ordered vector, and then extract the target keyword information in the first device information that is different from the reference information based on the comparison result, and save the target keyword information to the target dimension of the multidimensional ordered vector, and the multidimensional ordered vector is saved in the cache corresponding to the background server. In this case, the multidimensional ordered vector will not repeatedly store the same information as the reference information, thereby reducing the repeated information in the cache, reducing the occupation of the cache space by the device information, thereby reducing the storage pressure of the cache, and improving the storage efficiency in the cache.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本发明实施例的技术方案，下面将对本发明实施例中所需要使用的附图作简单的介绍，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following is a brief introduction to the drawings required for use in the embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
图1是本发明一个实施例提供的信息存储方法的流程示意图；FIG1 is a schematic flow chart of an information storage method provided by an embodiment of the present invention;
图2是本发明另一个实施例提供的信息存储方法的流程示意图；FIG2 is a flow chart of an information storage method provided by another embodiment of the present invention;
图3是本发明一个实施例提供的信息存储装置的结构示意图；FIG3 is a schematic diagram of the structure of an information storage device provided by an embodiment of the present invention;
图4是本发明一个实施例提供的服务器的结构示意图。FIG. 4 is a schematic diagram of the structure of a server provided by an embodiment of the present invention.
具体实施方式Detailed ways
下面将详细描述本发明的各个方面的特征和示例性实施例，为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及具体实施例，对本发明进行进一步详细描述。应理解，此处所描述的具体实施例仅被配置为解释本发明，并不被配置为限定本发明。对于本领域技术人员来说，本发明可以在不需要这些具体细节中的一些细节的情况下实施。下面对实施例的描述仅仅是为了通过示出本发明的示例来提供对本发明更好的理解。The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
如背景技术所述，目前后台服务器会周期性获取自身负责的多台设备的设备信息，并将设备信息存储至缓存，导致缓存需要存储的信息量非常大，存储压力大。As described in the background technology, currently the background server periodically obtains the device information of multiple devices that it is responsible for, and stores the device information in the cache, resulting in a very large amount of information that needs to be stored in the cache, resulting in high storage pressure.
为了解决现有技术问题，本发明实施例提供了一种信息存储方法、装置、服务器及介质。下面首先对本发明实施例所提供的信息存储方法进行介绍。参见图1，图1示出了本发明一个实施例提供的信息存储方法的流程示意图。该方法可以应用于服务器。In order to solve the problems of the prior art, an embodiment of the present invention provides an information storage method, device, server and medium. The information storage method provided by an embodiment of the present invention is first introduced below. Referring to FIG. 1, FIG. 1 shows a flow chart of an information storage method provided by an embodiment of the present invention. The method can be applied to a server.
S110，获取第i次采集到的第一设备的第一设备信息；1≤i≤N，N为缓存中保存的每个多维有序向量对应的预设采集次数；i和N均为正整数。S110, obtaining the first device information of the first device collected for the i-th time; 1≤i≤N, N is the preset number of collection times corresponding to each multidimensional ordered vector stored in the cache; i and N are both positive integers.
每个多维有序向量中能够保存N次采集的N个第一设备信息的目标关键值信息，例如N等于5。即本申请实施例会在N个不同的时间内采集第一设备的第一设备信息，其中，每连续N次采集的N个第一设备信息的目标关键值信息会保存至一个多维有序向量内。Each multidimensional ordered vector can store target key value information of N first device information collected N times, for example, N is equal to 5. That is, the embodiment of the present application will collect the first device information of the first device at N different times, wherein the target key value information of the N first device information collected N times in succession will be stored in a multidimensional ordered vector.
S120，将第一设备信息与多维有序向量中的基准信息进行比较，提取第一设备信息中与基准信息不同的目标关键值信息；其中，多维有序向量与第一设备对应。S120, comparing the first device information with the reference information in the multidimensional ordered vector, and extracting target key value information in the first device information that is different from the reference information; wherein the multidimensional ordered vector corresponds to the first device.
由于针对同一个多维有序向量的N个第一设备信息针对于同一个设备，且N个第一设备信息均是针对同一组采集项进行采集的，即N个第一设备信息中包含的信息内容的类型是相同的，例如N个第一设备信息均包含设备标识、归属地等信息，因此导致这N个第一设备信息非常相似，存在许多的重复信息，这些重复信息会浪费缓存的存储空间。Since the N first device information for the same multidimensional ordered vector are for the same device, and the N first device information are all collected for the same group of collection items, that is, the type of information content contained in the N first device information is the same, for example, the N first device information all contain information such as device identification and location, therefore, the N first device information are very similar and there is a lot of duplicate information, which will waste cache storage space.
因此，本实施例在多维有序向量中设置了一个基准信息，仅将第一设备信息中与基准信息不同的部分信息进行保存，即将各个第一设备信息中与基准信息不同的变化量进行保存，从而避免了多维有序向量中存储有较多重复信息的情况。并且这种情况下，将N个目标关键值信息与基准信息求和，即能够得到N个第一设备信息，因此这种方式也不会造成原始设备信息的丢失。Therefore, this embodiment sets a reference information in the multidimensional ordered vector, and only saves the part of the first device information that is different from the reference information, that is, saves the change amount in each first device information that is different from the reference information, thereby avoiding the situation where a lot of duplicate information is stored in the multidimensional ordered vector. In this case, by summing the N target key value information and the reference information, N first device information can be obtained, so this method will not cause the loss of the original device information.
S130，将目标关键值信息保存至多维有序向量中目标关键值信息对应的目标维度中。S130, saving the target key value information into the target dimension corresponding to the target key value information in the multi-dimensional ordered vector.
多维有序向量包括多个维度的向量值，为了保证向量的顺序性，会按照采集第一设备信息的时间顺序，依次将得到的目标关键值信息顺序填充至对应的目标维度内，例如多维有序向量中包括4个用于存储目标关键值信息的维度{D1,D2,D3,D4}，则后续将4次采集到的第一设备信息对应的目标关键值信息，依据D1-D2-D3-D4的顺序，依次存储至对应的维度内。其中，由于上述多维有序向量是有序的，因此需要采用Redis中的有序集合zset的结构进行存储。The multidimensional ordered vector includes vector values of multiple dimensions. In order to ensure the order of the vector, the target key value information obtained will be sequentially filled into the corresponding target dimension in the order of the time when the first device information is collected. For example, the multidimensional ordered vector includes 4 dimensions {D1, D2, D3, D4} for storing target key value information. Then, the target key value information corresponding to the first device information collected 4 times will be stored in the corresponding dimension in the order of D1-D2-D3-D4. Since the multidimensional ordered vector is ordered, it is necessary to use the structure of the ordered set zset in Redis for storage.
其中，在得到目标关键值信息后，为了将目标关键值信息保存至多维有序向量中的目标维度中，目标关键值信息会转换为向量的格式，因此，本实施例中将第一设备信息与基准信息比对，提取目标关键值信息保存至目标维度的操作，可以称为向量化操作。Among them, after obtaining the target key value information, in order to save the target key value information to the target dimension in the multidimensional ordered vector, the target key value information will be converted into a vector format. Therefore, in this embodiment, the operation of comparing the first device information with the benchmark information and extracting the target key value information and saving it to the target dimension can be called a vectorized operation.
上述向量化操作，由单个zset存储改为string类型存储基础设备信息(即基准信息)+zset存储变化量(即目标关键值信息)的方式，将zset的值变小，取得信息压缩的同时，使redis的结构优化收益可观。The above vectorized operation changes the storage of a single zset to a string type storage of basic device information (i.e., benchmark information) + zset storage of changes (i.e., target key value information), which reduces the value of zset, compresses information, and makes significant gains in redis structure optimization.
其中，还可以限制多维有序向量中，N条目标关键值信息的平均长度为1K。当然，也可以不限定N条目标关键值信息的平均长度，本发明对此不作限定。The average length of the N pieces of target key value information in the multidimensional ordered vector may also be limited to 1 K. Of course, the average length of the N pieces of target key value information may not be limited, and the present invention does not limit this.
本发明实施例的方法，在获取第i次采集到的第一设备的第一设备信息后，会将第一设备信息与多维有序向量中设置的基准信息进行比较，之后基于比较结果，提取出第一设备信息中与基准信息不同的目标关键字信息，并将目标关键字信息保存至多维有序向量的目标维度中，该多维有序向量保存于后台服务器对应的缓存中。这种情况下，使得多维有序向量中，不会重复存储与基准信息相同的信息，从而减少了缓存中的重复信息，减少了设备信息对缓存空间的占用，从而降低了缓存的存储压力，提高缓存中的存储效率。The method of the embodiment of the present invention, after obtaining the first device information of the first device collected for the i-th time, will compare the first device information with the reference information set in the multidimensional ordered vector, and then extract the target keyword information in the first device information that is different from the reference information based on the comparison result, and save the target keyword information to the target dimension of the multidimensional ordered vector, and the multidimensional ordered vector is saved in the cache corresponding to the background server. In this case, the multidimensional ordered vector will not repeatedly store the same information as the reference information, thereby reducing the repeated information in the cache, reducing the occupation of the cache space by the device information, thereby reducing the storage pressure of the cache, and improving the storage efficiency in the cache.
可选地，上述第一设备信息可以包括多个采集项信息以及与各个采集项信息一一对应多个第一关键值信息。这里的采集项信息指的是需要对第一设备进行采集的采集对象，例如归属地、用户账号等，第一关键值信息指的是采集项所对应的采集内容，例如归属地对应的第一关键值信息为：中国，用户账号对应的第一关键值信息为：135XXXXXXXX等。第一设备信息中包含采集项信息及其对应的第一关键值信息，能够方便确认每个第一关键值信息的定义，从而方便了后续的信息比对，避免出现信息比对错位的情况。Optionally, the first device information may include multiple collection item information and multiple first key value information corresponding to each collection item information. The collection item information here refers to the collection object that needs to be collected on the first device, such as the place of origin, user account, etc. The first key value information refers to the collection content corresponding to the collection item, for example, the first key value information corresponding to the place of origin is: China, and the first key value information corresponding to the user account is: 135XXXXXXXX, etc. The first device information contains the collection item information and its corresponding first key value information, which can facilitate the confirmation of the definition of each first key value information, thereby facilitating subsequent information comparison and avoiding the occurrence of information comparison misalignment.
其中，由于采集项信息与第一关键值信息为一一对应的关系，因此采集项信息与第一关键值信可以采用JSON中的键值对(k-v)存储结构进行存储。其中，采集项信息为key，第一关键值信息为value。此外，第一设备信息需要是能够拆分为C个域的信息，这种情况下，第一设备信息才能够采用k-v的格式进行设置。Among them, since the collection item information and the first key value information are in a one-to-one correspondence, the collection item information and the first key value information can be stored in a key-value pair (k-v) storage structure in JSON. Among them, the collection item information is the key and the first key value information is the value. In addition, the first device information needs to be information that can be split into C domains. In this case, the first device information can be set in the k-v format.
由于对于同一个系统来说，其获取设备信息的目的是相同的，因此这种情况下，该系统获取的全部设备信息中的采集项信息应该是都是相同的。这就导致在基准信息不包括采集项信息的情况下，每个目标关键值信息中具包含同一套采集项信息，即缓存中会重复存储采集项信息。Since the purpose of obtaining device information is the same for the same system, the collection item information in all device information obtained by the system should be the same. This results in that when the benchmark information does not include the collection item information, each target key value information contains the same set of collection item information, that is, the collection item information is stored repeatedly in the cache.
为了避免上述情况，可选地，上述第一设备信息也可以包括按照预设顺序排序的多个第一关键值信息；各个第一关键值信息与第一设备对应的各个采集项信息一一对应。In order to avoid the above situation, optionally, the above first device information may also include multiple first key value information sorted in a preset order; each first key value information corresponds one-to-one to each collection item information corresponding to the first device.
本实施例中，去掉了第一设备信息的JSON属性，并预先设置了预设顺序，该预设顺序中的不同位置分别对应各个采集项信息。后续通过将第一关键值信息按照预设顺序进行排序，即能够方便后续服务器根据排序顺序确定各个排序位置处的第一关键值信息对应的采集项信息。这种方式下，是的在基准信息不包括采集项信息的情况下，提取到的目标关键值信息中也不会包含采集项信息，从而避免了缓存中重复存储采集项信息的情况，进一步提高了缓存的存储效率。此外，为了方便服务器区分各个排序位置，在第一属性信息中各个第一关键值信息之间可以使用分隔符“\”进行划分。In this embodiment, the JSON attribute of the first device information is removed, and a preset order is set in advance. Different positions in the preset order correspond to each collection item information. Subsequently, by sorting the first key value information according to the preset order, it is convenient for the subsequent server to determine the collection item information corresponding to the first key value information at each sorting position according to the sorting order. In this way, when the benchmark information does not include the collection item information, the extracted target key value information will not include the collection item information, thereby avoiding the situation where the collection item information is repeatedly stored in the cache, and further improving the storage efficiency of the cache. In addition, in order to facilitate the server to distinguish each sorting position, the separator "\" can be used to divide each first key value information in the first attribute information.
可选地，上述采集项信息可以包括以下至少一项：基带版本、归属地、日期、网络协议、设备类型、设备版本、国际移动设备识别码、国际移动用户识别码、操作语言、移动服务提供商、操作系统类型、操作系统版本、屏幕分辨率、屏幕尺寸、序列号、服务集标识、时区。Optionally, the above-mentioned collection item information may include at least one of the following: baseband version, place of origin, date, network protocol, device type, device version, International Mobile Equipment Identity Code, International Mobile Subscriber Identity Code, operating language, mobile service provider, operating system type, operating system version, screen resolution, screen size, serial number, service set identifier, and time zone.
这里的采集项信息决定了第一设备信息所包含的信息内容，因此，除了用于表示设备身份的国际移动设备识别码等标识之外，采集项信息主要基于服务器获取设备信息后的需求进行设定。例如，若服务器利用第一设备信息进行设备身份识别，则第一设备信息内可以主要包含设备特征信息，例如设备类型、设备版本、屏幕分辨率、屏幕尺寸等；若服务器利用第一设备信息进行信息推荐，则第一设备信息可以包括浏览记录、已安装的应用程序等。本发明并不限定采集项信息的具体内容和数量。The collected item information here determines the information content contained in the first device information. Therefore, in addition to the identifiers such as the International Mobile Equipment Identity Code used to indicate the device identity, the collected item information is mainly set based on the needs of the server after obtaining the device information. For example, if the server uses the first device information to identify the device identity, the first device information may mainly include device feature information, such as device type, device version, screen resolution, screen size, etc.; if the server uses the first device information to recommend information, the first device information may include browsing history, installed applications, etc. The present invention does not limit the specific content and quantity of the collected item information.
在一些实施例中，上述多维有序向量可以包括M个维度，M＝N+2；多维有序向量中的两个维度预先存储有基准信息以及第一设备的设备标识。In some embodiments, the multidimensional ordered vector may include M dimensions, where M=N+2; two dimensions of the multidimensional ordered vector pre-store the reference information and the device identification of the first device.
在本实施例中，假设N＝4，则多维有序向量的结构为{K1,Vbase,D1,D2,D3,D4}，其中K1为存储设备标识的维度，Vbase为存储基准信息的维度，D1～D4为存储目标关键值信息的维度。这种情况下，基准信息为预先设定的信息，用户可以根据第一设备的的特征以及用户账号的情况进行设置，基准信息设置的自由度更高，用户可以根据获取设备信息的目的，来调整基准信息的内容，并且能够尽可能避免多维有序向量中存在重复信息。In this embodiment, assuming that N=4, the structure of the multidimensional ordered vector is {K1, Vbase, D1, D2, D3, D4}, where K1 is the dimension for storing device identification, Vbase is the dimension for storing benchmark information, and D1-D4 are the dimensions for storing target key value information. In this case, the benchmark information is pre-set information, and the user can set it according to the characteristics of the first device and the situation of the user account. The freedom of setting the benchmark information is higher, and the user can adjust the content of the benchmark information according to the purpose of obtaining device information, and can avoid duplicate information in the multidimensional ordered vector as much as possible.
在另一些实施例中，上述多维有序向量可以包括M个维度，M＝N+1；多维有序向量中的一个维度预先存储有第一设备的设备标识；In some other embodiments, the multidimensional ordered vector may include M dimensions, where M=N+1; one dimension of the multidimensional ordered vector pre-stores a device identification of the first device;
在i＝1的情况下，将第一设备信息作为基准信息，并将基准信息保存至多维有序向量中基准信息对应的维度中。In the case of i=1, the first device information is used as the reference information, and the reference information is stored in the dimension corresponding to the reference information in the multi-dimensional ordered vector.
本实施例中，假设N＝4，则多维有序向量的结构为{K1,Vbase,D1,D2,D3}，其中，将第一次采集的第一设备信息作为该多维有序向量的基准信息。这种方式下，基准信息不需要用户进行单独设置，而是直接利用所采集的设备信息即可，简化了用户的操作。其中，第1次采集的第一设备信息作为基准信息时，基准信息与其他第一设备信息之间的差异信息的长度为D[N-1]，D[N-1]可以限定为<128，其中，任一采集项对应的信息内容不同即可将第一设备的整体长度作为D[N-1]。In this embodiment, assuming that N=4, the structure of the multidimensional ordered vector is {K1, Vbase, D1, D2, D3}, wherein the first device information collected for the first time is used as the baseline information of the multidimensional ordered vector. In this way, the baseline information does not need to be set separately by the user, but can be directly used by the collected device information, which simplifies the user's operation. Among them, when the first device information collected for the first time is used as the baseline information, the length of the difference information between the baseline information and the other first device information is D[N-1], and D[N-1] can be limited to <128, wherein the overall length of the first device can be used as D[N-1] if the information content corresponding to any collection item is different.
由于目前redis所使用的底层数据编码类型存在优化空间，基于此，在其他一些实施例中，S120之后，S130之前，该方法还可以包括：Since there is room for optimization of the underlying data encoding type currently used by redis, based on this, in some other embodiments, after S120 and before S130, the method may further include:
对目标关键值信息中满足预设编码条件的第一字段信息进行压缩编码，得到编码后的目标关键值信息；Compressing and encoding the first field information that meets the preset encoding condition in the target key value information to obtain the encoded target key value information;
上述S130可以包括：The above S130 may include:
将编码后的目标关键值信息保存至多维有序向量中的目标维度中。The encoded target key value information is saved in the target dimension in the multidimensional ordered vector.
本实施例中，将目标关键值信息中，满足预设编码条件的第一字段信息进行了压缩编码，从而从编码类型的方向上，压缩了目标关键值信息的数据大小，从而进一步降低了目标关键值信息对存储空间的占用。In this embodiment, the first field information in the target key value information that meets the preset encoding conditions is compressed and encoded, thereby compressing the data size of the target key value information in terms of the encoding type, thereby further reducing the storage space occupied by the target key value information.
可选地，压缩编码为霍夫曼编码，上述预设编码条件可以包括：Optionally, the compression coding is Huffman coding, and the above preset coding conditions may include:
字段信息具有对应的取值范围，且字段信息基于设备属性的概率分布的稳定性满足预设稳定条件。The field information has a corresponding value range, and the stability of the probability distribution of the field information based on the device attribute meets the preset stability condition.
在计算机数据处理中，霍夫曼编码使用变长编码表对源符号(如文件中的一个字母)进行编码，其中变长编码表是通过一种评估来源符号出现机率的方法得到的，出现机率高的字母使用较短的编码，反之出现机率低的则使用较长的编码，这便使编码之后的字符串的平均长度、期望值降低，从而达到无损压缩数据的目的。霍夫曼编码针对取值空间有限的字段且该字段基于设备属性的概率分布需要满足大体稳定的条件，针对满足以上两个条件的字段信息进行霍夫曼编码，取得逼近理论的最短编码长度(信息熵大小)，从而实现压缩字段信息的目的。In computer data processing, Huffman coding uses a variable-length coding table to encode source symbols (such as a letter in a file), where the variable-length coding table is obtained by a method of evaluating the probability of occurrence of source symbols. Letters with high probability of occurrence use shorter codes, while letters with low probability of occurrence use longer codes. This reduces the average length and expected value of the encoded string, thereby achieving the purpose of lossless data compression. Huffman coding is aimed at fields with limited value space and the probability distribution of the field based on device attributes needs to meet the condition of being generally stable. Huffman coding is performed on field information that meets the above two conditions to obtain the shortest code length (information entropy size) of the approximation theory, thereby achieving the purpose of compressing field information.
此外，该方法还可以对基准信息中满足上述预设编码条件的字段信息进行压缩编码，从而减少基准信息的存储大小。In addition, the method can also compress and encode the field information in the reference information that meets the above-mentioned preset encoding conditions, thereby reducing the storage size of the reference information.
可选的，上述压缩编码方式可以包括：对目标关键值信息中，类型有限的列进行压缩编码。例如，这里以“时区”对应的一列进行介绍，本例中有两种GMT+09:00和GMT+08:00，压缩编码方式可以利用信息熵公式进行计算；故在字符串中只需要一个char即可，例如，编码时区东八区为64，对应ASCII码表A，东九区为65，对应码表为B，编码后的信息仅占1字节，相较于编码前需要占用的9字节，节约了8字节；若使用二进制编码方式，则该列信息可以由9字节的72bit缩减为5bit。Optionally, the above compression encoding method may include: compressing and encoding columns of limited types in the target key value information. For example, here we introduce a column corresponding to "time zone". In this example, there are two types of GMT+09:00 and GMT+08:00. The compression encoding method can be calculated using the information entropy formula; therefore, only one char is needed in the string. For example, the encoding time zone East 8 is 64, corresponding to ASCII code table A, East 9 is 65, corresponding to code table B, and the encoded information only occupies 1 byte, which saves 8 bytes compared to the 9 bytes required before encoding; if binary encoding is used, the information in this column can be reduced from 9 bytes of 72 bits to 5 bits.
对于本例目标关键值信息：For this example, the target key value information is:
',40:31:3c:1e:40:6d,101.228.28.99,,,,,,,,,,,,,,,,,,,,,,"Xiaomi_406C",GMT+09:00,,,'',40:31:3c:1e:40:6d,101.228.28.99,,,,,,,,,,,,,,,,,,,,,,"Xiaomi_406C",GMT+09:00,, ,'
进行压缩编码后得到：After compression encoding, we get:
',40:31:3c:1e:40:6d,101.228.28.99,,,,,,,,,,,,,,,,,,,,,,"Xiaomi_406C",B,,,'',40:31:3c:1e:40:6d,101.228.28.99,,,,,,,,,,,,,,,,,,,,,,,"Xiaomi_406C",B,,,'
本实施例中进行压缩编码，是为了获得更小的目标关键值信息长度，更小的目标关键值信息长度可以取得更大的压缩比率，进一步降低对缓存空间的占用，也可以减小对读写性能的影响，对于多维有序向量中类型有限的列都可以做这样的压缩编码操作。压缩编码方式具体如何使用要以最终的业务场景对读写性能的要求而取舍。In this embodiment, compression coding is performed to obtain a smaller target key value information length. A smaller target key value information length can achieve a greater compression ratio, further reduce the occupancy of cache space, and also reduce the impact on read and write performance. Such compression coding operations can be performed on columns of limited types in multidimensional ordered vectors. How to use the compression coding method specifically depends on the requirements of the final business scenario for read and write performance.
当然，在其他实施例中，也可以采用其他类型的压缩编码。Of course, in other embodiments, other types of compression encoding may also be used.
由于当前存储同一个设备的多条设备信息时，本身结构采用json存储，且需要有序性及集合set的不重复属性，故选用redis的有序集合(zset)作为存储结构，针对redis的zset结构，设备对应的采集项会比较多，导致设备信息中的值value长度较大，在1-2k之间，基于此，为了避免较长的设备信息对读写性能的影响，目前redis的底层存储结构使用的是跳表(skiplist)的结构，但是这种结构会对缓存空间造成较大的占用。When storing multiple pieces of device information of the same device, the structure itself is stored in JSON, and orderliness and the non-repetitive properties of the set are required. Therefore, the ordered set (zset) of redis is selected as the storage structure. For the zset structure of redis, there will be more collection items corresponding to the device, resulting in a longer value length in the device information, between 1-2k. Based on this, in order to avoid the impact of longer device information on read and write performance, the underlying storage structure of redis currently uses a skiplist structure, but this structure will occupy a large amount of cache space.
基于此，在本发明再一些实施例中，上述S130可以包括：Based on this, in some further embodiments of the present invention, the above S130 may include:
采用压缩表的结构，将目标关键值信息保存至多维有序向量中的目标维度中。The structure of the compression table is adopted to save the target key value information into the target dimension in the multi-dimensional ordered vector.
由于本发明的前述实施例中，通过与基准信息比较以及提取目标关键值信息，从而减小了目标关键值信息的长度，这种情况下，采用压缩表(ziplist)的结构进行存储，即能够尽可能避免压缩表中字段长度过长所造成的的读写性能下降严重的情况，从而使得在降低缓存空间压力的同时，也能够尽可能保证读写性能的实时性。In the aforementioned embodiments of the present invention, the length of the target key value information is reduced by comparing with the benchmark information and extracting the target key value information. In this case, a compressed table (ziplist) structure is used for storage, which can avoid as much as possible the serious degradation of the read and write performance caused by the excessive length of the fields in the compressed table. This can reduce the cache space pressure while ensuring the real-time nature of the read and write performance as much as possible.
为了在存储优化的同时，也能够尽可能保证读写性能的实时性的，本实施例在对第一设备信息进行信息缩减后，再利用压缩表对得到的目标关键值信息进行存储，而想要避免压缩表的结构对读写性能造成影响，则需要保证被压缩的字段信息的长度小于L，其中zset-max-ziplist-value＝L。但是由于目标关键值信息中，无法保证全部的字段信息均小于L，例如部分采集项对应的信息较长的情况，则这部分字段信息若采用压缩表的结构进行存储的话，则可能会对读写性能造成影响。In order to optimize storage and ensure the real-time performance of read and write performance as much as possible, this embodiment uses a compression table to store the obtained target key value information after reducing the first device information. In order to avoid the structure of the compression table from affecting the read and write performance, it is necessary to ensure that the length of the compressed field information is less than L, where zset-max-ziplist-value = L. However, since it is impossible to ensure that all field information in the target key value information is less than L, for example, if the information corresponding to some collection items is long, if this part of the field information is stored using the structure of a compression table, it may affect the read and write performance.
基于此，在本发明又一些实施例中，上述S130可以包括：Based on this, in some further embodiments of the present invention, the above S130 may include:
确定目标关键值信息中，字段信息的长度小于或等于预设长度参数L的第二字段信息占目标关键值信息的比例；Determine the proportion of second field information whose length of field information is less than or equal to a preset length parameter L in the target key value information;
在比例达到预设比例的情况下，采用压缩表的结构将第二字段信息保存至多维有序向量中的目标维度中，并采用跳表的结构将目标关键字信息中除第二字段信息之外的第三字段信息保存至多维有序向量中的目标维度中。When the ratio reaches the preset ratio, the second field information is saved in the target dimension in the multidimensional ordered vector using the compression table structure, and the third field information in the target keyword information except the second field information is saved in the target dimension in the multidimensional ordered vector using the jump table structure.
在比例未达到预设比例的情况下，采用跳表的结构将目标关键字信息保存至多维有序向量中的目标维度中。When the ratio does not reach the preset ratio, a skip table structure is used to save the target keyword information into the target dimension in the multi-dimensional ordered vector.
本实施例中，首先确定了目标关键值信息中，哪些字段信息的长度小于或等于L，之后将这些字段信息占目标关键值信息的比例与预设比例进行比较，例如预设比例可以为99.9％，在比例达到预设比例的情况下，表明此时目标关键值信息中大部分字段信息以及满足采用压缩表结构的条件了，因此，可以将这部分字段信息转换为压缩表，而剩余少部分长度大于L的字段信息，则可以采用跳表继续保存。In this embodiment, it is first determined which field information in the target key value information has a length less than or equal to L, and then the proportion of these field information in the target key value information is compared with a preset proportion. For example, the preset proportion can be 99.9%. When the proportion reaches the preset proportion, it indicates that most of the field information in the target key value information meets the conditions for using a compressed table structure. Therefore, this part of the field information can be converted into a compressed table, and the remaining small part of the field information whose length is greater than L can be saved using a jump table.
这种情况下，通过修改L的值，即能够将绝大部分字段信息的存储结构，从跳表变为压缩表，通过压缩表的结构来实现较好的存储结构收益，并且，由于字段信息的长度不大于L，也能够保证读写的实时性，避免较长的字段信息采用压缩表存储时对读写性能的影响。而在比例未达到预设比例的情况下，表明此时目标关键值信息中包括较多的字段信息的长度不符合压缩表的要求，因此，这种情况下，可以直接采用跳表进行存储，从而优先保证读写性能不受影响。其中，L可以取128以下的数值。In this case, by modifying the value of L, the storage structure of most field information can be changed from a skip table to a compressed table, and a better storage structure benefit can be achieved through the structure of the compressed table. Moreover, since the length of the field information is not greater than L, the real-time reading and writing can be guaranteed, and the impact of longer field information on the reading and writing performance when stored in a compressed table is avoided. If the ratio does not reach the preset ratio, it indicates that the length of more field information included in the target key value information does not meet the requirements of the compressed table. Therefore, in this case, a skip table can be directly used for storage, thereby giving priority to ensuring that the reading and writing performance is not affected. Among them, L can take a value below 128.
在另一些实施例中，上述S130可以包括：In some other embodiments, the above S130 may include:
确定目标关键值信息中，字段信息的长度小于或等于预设长度参数L的第二字段信息；Determine second field information in the target key value information, the length of which is less than or equal to a preset length parameter L;
采用压缩表的结构将第二字段信息保存至多维有序向量中的目标维度中，并采用跳表的结构将目标关键字信息中除第二字段信息之外的第三字段信息保存至多维有序向量中的目标维度中。The second field information is saved in the target dimension of the multidimensional ordered vector using a compression table structure, and the third field information except the second field information in the target keyword information is saved in the target dimension of the multidimensional ordered vector using a skip table structure.
本实施例中，通过字段信息的长度是否大于L，来将目标关键值信息划分为第二字段信息和第三字段信息，之后对于长度不大于L的第二字段信息通过压缩表的结构存储，对长度大于L的第三字段信息通过跳表的结构存储。这种方式下，能够保证在对redis的存储效率进行优化的同时尽量减小对读写操作实时性的影响。In this embodiment, the target key value information is divided into the second field information and the third field information according to whether the length of the field information is greater than L. Then, the second field information whose length is not greater than L is stored in a compression table structure, and the third field information whose length is greater than L is stored in a skip table structure. In this way, it is possible to optimize the storage efficiency of redis while minimizing the impact on the real-time performance of read and write operations.
由于多维有序向量中存储的相似设备信息的数量N，决定了读写效率下降的程度，对于跳表编码，读写的复杂度为O(logN)，而压缩表编码中，读平均复杂度为o(N^2)，写平均复杂度为o(N)。因此，为了降低压缩表编码的读写复杂度，满足业务的实时性需求，N应该设置为较小的值，可选地，N为小于10的正整数。当然，以上仅为一种具体实现方式，本发明不限定N的具体数值以及取值范围。Since the number N of similar device information stored in the multidimensional ordered vector determines the degree of decrease in read and write efficiency, for jump table encoding, the complexity of read and write is O(logN), while in compressed table encoding, the average read complexity is o(N^2) and the average write complexity is o(N). Therefore, in order to reduce the read and write complexity of compressed table encoding and meet the real-time requirements of the business, N should be set to a smaller value. Optionally, N is a positive integer less than 10. Of course, the above is only a specific implementation method, and the present invention does not limit the specific value and value range of N.
基于前述实施例，本申请还提供了另一种信息存储方法，参见图2，图2是本发明另一个实施例提供的信息存储方法的流程示意图。该方法包括：Based on the above embodiment, the present application also provides another information storage method, see Figure 2, which is a flow chart of the information storage method provided by another embodiment of the present invention. The method includes:
S210，获取第i次采集到的第一设备的第一设备信息；第一设备信息包括按照预设顺序排序的多个第一关键值信息。S210, obtaining first device information of a first device collected for an i-th time; the first device information includes a plurality of first key value information sorted in a preset order.
S220，将第一设备信息与多维有序向量中的基准信息进行比较，提取第一设备信息中与基准信息不同的目标关键值信息。S220: Compare the first device information with the reference information in the multi-dimensional ordered vector, and extract target key value information in the first device information that is different from the reference information.
S230，对目标关键值信息中满足预设编码条件的第一字段信息进行压缩编码，得到编码后的目标关键值信息。S230, compress and encode the first field information in the target key value information that meets the preset encoding condition to obtain the encoded target key value information.
S240，确定编码后的目标关键值信息中，字段信息的长度小于或等于预设长度参数L的第二字段信息占目标关键值信息的比例。S240, determining the proportion of second field information whose length of field information is less than or equal to a preset length parameter L in the encoded target key value information to the target key value information.
S250，在比例达到预设比例的情况下，采用压缩表的结构将第二字段信息保存至多维有序向量中的目标维度中，并采用跳表的结构将目标关键字信息中除第二字段信息之外的第三字段信息保存至多维有序向量中的目标维度中。S250, when the ratio reaches a preset ratio, a compression table structure is used to save the second field information into a target dimension in a multidimensional ordered vector, and a jump table structure is used to save the third field information in the target keyword information except the second field information into a target dimension in a multidimensional ordered vector.
本实施例中，为了兼顾存储优化及减小读写实时性的目的，首先对相似的设备信息进行向量化，得到去除重复信息后的目标关键值信息；然后对目标关键值信息进行编码压缩，再次减小目标关键值信息的数据大小，将目标关键值信息中的大多数字段信息的长度控制在L以下；之后对编码后的目标关键值信息进行存储结构优化，利用ziplist替换skiplist对长度在L之下的字段信息进行存储，使得在得到redis的存储空间优化的同时尽量减小对读写操作实时性的影响。In this embodiment, in order to take into account both storage optimization and reducing the real-time performance of reading and writing, similar device information is first vectorized to obtain the target key value information after removing duplicate information; then the target key value information is encoded and compressed to reduce the data size of the target key value information again, and the length of most field information in the target key value information is controlled below L; then the storage structure of the encoded target key value information is optimized, and ziplist is used to replace skiplist to store field information with a length below L, so that the storage space of redis is optimized while minimizing the impact on the real-time performance of read and write operations.
基于与前述方法实施例相同的发明构思，下面首先对本发明实施例所提供的信息存储装置进行介绍。参见图3，图3示出了本发明一个实施例提供的信息存储装置的结构示意图；该装置包括：Based on the same inventive concept as the above method embodiment, the information storage device provided by the embodiment of the present invention is first introduced below. Referring to FIG. 3 , FIG. 3 shows a schematic diagram of the structure of the information storage device provided by an embodiment of the present invention; the device includes:
信息获取模块310，用于获取第i次采集到的第一设备的第一设备信息；1≤i≤N，N为缓存中保存的每个多维有序向量对应的预设采集次数；i和N均为正整数；The information acquisition module 310 is used to acquire the first device information of the first device acquired for the i-th time; 1≤i≤N, N is the preset number of acquisitions corresponding to each multidimensional ordered vector stored in the cache; i and N are both positive integers;
信息提取模块320，用于将第一设备信息与多维有序向量中的基准信息进行比较，提取第一设备信息中与基准信息不同的目标关键值信息；其中，多维有序向量与第一设备对应；An information extraction module 320, configured to compare the first device information with the reference information in the multidimensional ordered vector, and extract target key value information in the first device information that is different from the reference information; wherein the multidimensional ordered vector corresponds to the first device;
第一保存模块330，用于将目标关键值信息保存至多维有序向量中目标关键值信息对应的目标维度中。The first saving module 330 is used to save the target key value information into the target dimension corresponding to the target key value information in the multi-dimensional ordered vector.
本发明实施例的方法，在获取第i次采集到的第一设备的第一设备信息后，会将第一设备信息与多维有序向量中设置的基准信息进行比较，之后基于比较结果，提取出第一设备信息中与基准信息不同的目标关键字信息，并将目标关键字信息保存至多维有序向量的目标维度中，该多维有序向量保存于后台服务器对应的缓存中。这种情况下，使得多维有序向量中，不会重复存储与基准信息相同的信息，从而减少了缓存中的重复信息，减少了设备信息对缓存空间的占用，从而降低了缓存的存储压力，提高缓存中的存储效率。The method of the embodiment of the present invention, after obtaining the first device information of the first device collected for the i-th time, will compare the first device information with the reference information set in the multidimensional ordered vector, and then extract the target keyword information in the first device information that is different from the reference information based on the comparison result, and save the target keyword information to the target dimension of the multidimensional ordered vector, and the multidimensional ordered vector is saved in the cache corresponding to the background server. In this case, the multidimensional ordered vector will not repeatedly store the same information as the reference information, thereby reducing the repeated information in the cache, reducing the occupation of the cache space by the device information, thereby reducing the storage pressure of the cache, and improving the storage efficiency in the cache.
在一些实施例中，该装置还可以包括：In some embodiments, the apparatus may further include:
编码模块，用于对所述目标关键值信息中满足预设编码条件的第一字段信息进行压缩编码，得到编码后的所述目标关键值信息；An encoding module, used for compressing and encoding the first field information in the target key value information that meets the preset encoding condition to obtain the encoded target key value information;
第一保存模块330可以用于：将编码后的所述目标关键值信息保存至所述多维有序向量中的所述目标维度中。The first saving module 330 may be used to save the encoded target key value information into the target dimension in the multi-dimensional ordered vector.
本实施例中，将目标关键值信息中，满足预设编码条件的第一字段信息进行了压缩编码，从而从编码类型的方向上，压缩了目标关键值信息的数据大小，从而进一步降低了目标关键值信息对存储空间的占用。In this embodiment, the first field information in the target key value information that meets the preset encoding conditions is compressed and encoded, thereby compressing the data size of the target key value information in terms of the encoding type, thereby further reducing the storage space occupied by the target key value information.
在另一些实施例中，上述第一保存模块330可以用于：In some other embodiments, the first saving module 330 may be used to:
采用压缩表的结构，将所述目标关键值信息保存至所述多维有序向量中的所述目标维度中。The target key value information is stored in the target dimension in the multi-dimensional ordered vector using a compression table structure.
由于本发明的前述实施例中，通过与基准信息比较以及提取目标关键值信息，从而减小了目标关键值信息的长度，这种情况下，采用压缩表(ziplist)的结构进行存储，即能够尽可能避免压缩表中字段长度过长所造成的的读写性能下降严重的情况，从而使得在降低缓存空间压力的同时，也能够尽可能保证读写性能的实时性。In the aforementioned embodiments of the present invention, the length of the target key value information is reduced by comparing with the benchmark information and extracting the target key value information. In this case, a compressed table (ziplist) structure is used for storage, which can avoid as much as possible the serious degradation of the read and write performance caused by the excessive length of the fields in the compressed table. This can reduce the cache space pressure while ensuring the real-time nature of the read and write performance as much as possible.
在再一些实施例中，上述第一保存模块330可以包括：In some further embodiments, the first saving module 330 may include:
比例确定单元，用于确定所述目标关键值信息中，字段信息的长度小于或等于预设长度参数L的第二字段信息占所述目标关键值信息的比例；A ratio determination unit, configured to determine the ratio of second field information whose length of field information is less than or equal to a preset length parameter L in the target key value information to the target key value information;
第一保存单元，用于在所述比例达到预设比例的情况下，采用压缩表的结构将所述第二字段信息保存至所述多维有序向量中的所述目标维度中，并采用跳表的结构将所述目标关键字信息中除所述第二字段信息之外的第三字段信息保存至所述多维有序向量中的所述目标维度中。The first saving unit is used to save the second field information to the target dimension in the multidimensional ordered vector using a compression table structure when the ratio reaches a preset ratio, and to save the third field information in the target keyword information except the second field information to the target dimension in the multidimensional ordered vector using a jump table structure.
本实施例中，首先确定了目标关键值信息中，哪些字段信息的长度小于或等于L，之后将这些字段信息占目标关键值信息的比例与预设比例进行比较，例如预设比例可以为99.9％，在比例达到预设比例的情况下，表明此时目标关键值信息中大部分字段信息以及满足采用压缩表结构的条件了，因此，可以将这部分字段信息转换为压缩表，而剩余少部分长度大于L的字段信息，则可以采用跳表继续保存。In this embodiment, it is first determined which field information in the target key value information has a length less than or equal to L, and then the proportion of these field information in the target key value information is compared with a preset proportion. For example, the preset proportion can be 99.9%. When the proportion reaches the preset proportion, it indicates that most of the field information in the target key value information meets the conditions for using a compressed table structure. Therefore, this part of the field information can be converted into a compressed table, and the remaining small part of the field information whose length is greater than L can be saved using a jump table.
这种情况下，通过修改L的值，即能够将绝大部分字段信息的存储结构，从跳表变为压缩表，通过压缩表的结构来实现较好的存储结构收益，并且，由于字段信息的长度不大于L，也能够保证读写的实时性，避免较长的字段信息采用压缩表存储时对读写性能的影响。而在比例未达到预设比例的情况下，表明此时目标关键值信息中包括较多的字段信息的长度不符合压缩表的要求，因此，这种情况下，可以直接采用跳表进行存储，从而优先保证读写性能不受影响。In this case, by modifying the value of L, the storage structure of most field information can be changed from a skip table to a compressed table, and a better storage structure benefit can be achieved through the structure of the compressed table. Moreover, since the length of the field information is not greater than L, the real-time reading and writing can be guaranteed, and the impact of longer field information on the reading and writing performance when stored in a compressed table is avoided. If the ratio does not reach the preset ratio, it indicates that the length of more field information included in the target key value information does not meet the requirements of the compressed table. Therefore, in this case, a skip table can be directly used for storage, thereby giving priority to ensuring that the reading and writing performance is not affected.
在又一些实施例中，上述第一保存模块330可以包括：In some further embodiments, the first saving module 330 may include:
字段确定单元，用于确定目标关键值信息中，字段信息的长度小于或等于预设长度参数L的第二字段信息；A field determination unit, used to determine second field information in the target key value information, the length of which is less than or equal to a preset length parameter L;
第二保存单元，用于采用压缩表的结构将第二字段信息保存至多维有序向量中的目标维度中，并采用跳表的结构将目标关键字信息中除第二字段信息之外的第三字段信息保存至多维有序向量中的目标维度中。The second saving unit is used to save the second field information into the target dimension in the multidimensional ordered vector by using the compression table structure, and save the third field information in the target keyword information except the second field information into the target dimension in the multidimensional ordered vector by using the skip table structure.
本实施例中，通过字段信息的长度是否大于L，来将目标关键值信息划分为第二字段信息和第三字段信息，之后对于长度不大于L的第二字段信息通过压缩表的结构存储，对长度大于L的第三字段信息通过跳表的结构存储。这种方式下，能够保证在对redis的存储效率进行优化的同时尽量减小对读写操作实时性的影响。In this embodiment, the target key value information is divided into the second field information and the third field information according to whether the length of the field information is greater than L. Then, the second field information whose length is not greater than L is stored in a compression table structure, and the third field information whose length is greater than L is stored in a skip table structure. In this way, it is possible to optimize the storage efficiency of redis while minimizing the impact on the real-time performance of read and write operations.
本发明实施例提供的信息存储装置能够实现图1-2的方法实施例中实现的各个过程，为避免重复，这里不再赘述。The information storage device provided in the embodiment of the present invention can implement each process implemented in the method embodiment of Figures 1-2, and will not be described again here to avoid repetition.
本发明实施例还提供了一种服务器，参见图4，图4是本发明一个实施例提供的服务器的结构示意图。An embodiment of the present invention further provides a server. See FIG. 4 , which is a schematic diagram of the structure of a server provided by an embodiment of the present invention.
上述服务器可以包括处理器401以及存储有计算机程序指令的存储器402。处理器401通过读取并执行存储器402中存储的计算机程序指令，以实现上述实施例中的任意一种信息存储方法。The server may include a processor 401 and a memory 402 storing computer program instructions. The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement any one of the information storage methods in the above embodiments.
具体地，上述处理器401可以包括中央处理器(CPU)，或者特定集成电路(Application Specific Integrated Circuit，ASIC)，或者可以被配置成实施本发明实施例的一个或多个集成电路。Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.
存储器402可以包括用于数据或指令的大容量存储器。举例来说而非限制，存储器402可包括硬盘驱动器(Hard Disk Drive，HDD)、软盘驱动器、闪存、光盘、磁光盘、磁带或通用串行总线(Universal Serial Bus，USB)驱动器或者两个或更多个以上这些的组合。在合适的情况下，存储器402可包括可移除或不可移除(或固定)的介质。在合适的情况下，存储器402可在综合网关容灾设备的内部或外部。在特定实施例中，存储器402是非易失性固态存储器。在特定实施例中，存储器402包括只读存储器(ROM)。在合适的情况下，该ROM可以是掩模编程的ROM、可编程ROM(PROM)、可擦除PROM(EPROM)、电可擦除PROM(EEPROM)、电可改写ROM(EAROM)或闪存或者两个或更多个以上这些的组合。The memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 402 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 402 is a non-volatile solid-state memory. In a specific embodiment, the memory 402 includes a read-only memory (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.
在一个示例中，服务器还可包括通信接口403和总线410。其中，如图4所示，处理器401、存储器402、通信接口403通过总线410连接并完成相互间的通信。In one example, the server may further include a communication interface 403 and a bus 410. As shown in Fig. 4, the processor 401, the memory 402, and the communication interface 403 are connected via the bus 410 and communicate with each other.
通信接口403，主要用于实现本发明实施例中各模块、装置、单元和/或设备之间的通信。The communication interface 403 is mainly used to implement the communication between the modules, devices, units and/or equipment in the embodiment of the present invention.
总线410包括硬件、软件或两者，将在线数据流量计费设备的部件彼此耦接在一起。举例来说而非限制，总线可包括加速图形端口(AGP)或其他图形总线、增强工业标准架构(EISA)总线、前端总线(FSB)、超传输(HT)互连、工业标准架构(ISA)总线、无限带宽互连、低引脚数(LPC)总线、存储器总线、微信道架构(MCA)总线、外围组件互连(PCI)总线、PCI-Express(PCI-X)总线、串行高级技术附件(SATA)总线、视频电子标准协会局部(VLB)总线或其他合适的总线或者两个或更多个以上这些的组合。在合适的情况下，总线410可包括一个或多个总线。尽管本发明实施例描述和示出了特定的总线，但本发明考虑任何合适的总线或互连。Bus 410 includes hardware, software or both, and the components of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.
另外，结合上述实施例中的信息存储方法，本发明实施例可提供一种计算机存储介质来实现。该计算机存储介质上存储有计算机程序指令；该计算机程序指令被处理器执行时实现上述实施例中的任意一种信息存储方法。In addition, in combination with the information storage method in the above embodiment, the embodiment of the present invention can provide a computer storage medium to implement. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the information storage methods in the above embodiment is implemented.
以上所述的结构框图中所示的功能块可以实现为硬件、软件、固件或者它们的组合。当以硬件方式实现时，其可以例如是电子电路、专用集成电路(ASIC)、适当的固件、插件、功能卡等等。当以软件方式实现时，本发明的元素是被用于执行所需任务的程序或者代码段。程序或者代码段可以存储在机器可读介质中，或者通过载波中携带的数据信号在传输介质或者通信链路上传送。“机器可读介质”可以包括能够存储或传输信息的任何介质。机器可读介质的例子包括电子电路、半导体存储器设备、ROM、闪存、可擦除ROM(EROM)、软盘、CD-ROM、光盘、硬盘、光纤介质、射频(RF)链路，等等。代码段可以经由诸如因特网、内联网等的计算机网络被下载。The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
上面参考根据本发明的实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各方面。应当理解，流程图和/或框图中的每个方框以及流程图和/或框图中各方框的组合可以由计算机程序指令实现。这些计算机程序指令可被提供给通用计算机、专用计算机、或其它可编程数据处理装置的处理器，以产生一种机器，使得经由计算机或其它可编程数据处理装置的处理器执行的这些指令使能对流程图和/或框图的一个或多个方框中指定的功能/动作的实现。这种处理器可以是但不限于是通用处理器、专用处理器、特殊应用处理器或者现场可编程逻辑电路。还可理解，框图和/或流程图中的每个方框以及框图和/或流程图中的方框的组合，也可以由执行指定的功能或动作的专用硬件来实现，或可由专用硬件和计算机指令的组合来实现。Aspects of the present invention are described above with reference to the flowchart and/or block diagram of the method, device (system) and computer program product according to an embodiment of the present invention. It should be understood that each box in the flowchart and/or block diagram and the combination of each box in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function/action specified in one or more boxes of the flowchart and/or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and/or flowchart and the combination of boxes in the block diagram and/or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
还需要说明的是，本发明中提及的示例性实施例，基于一系列的步骤或者装置描述一些方法或系统。但是，本发明不局限于上述步骤的顺序，也就是说，可以按照实施例中提及的顺序执行步骤，也可以不同于实施例中的顺序，或者若干步骤同时执行。It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.
以上所述，仅为本发明的具体实施方式，所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的系统、模块和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。应理解，本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本发明的保护范围之内。The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010955521.8ACN112181869B (en) | 2020-09-11 | 2020-09-11 | Information storage method, device, server and medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010955521.8ACN112181869B (en) | 2020-09-11 | 2020-09-11 | Information storage method, device, server and medium |
| Publication Number | Publication Date |
|---|---|
| CN112181869A CN112181869A (en) | 2021-01-05 |
| CN112181869Btrue CN112181869B (en) | 2024-07-19 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010955521.8AActiveCN112181869B (en) | 2020-09-11 | 2020-09-11 | Information storage method, device, server and medium |
| Country | Link |
|---|---|
| CN (1) | CN112181869B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115431901A (en)* | 2022-02-18 | 2022-12-06 | 北京罗克维尔斯科技有限公司 | A signal acquisition method, device, electronic equipment and storage medium |
| CN116567097B (en)* | 2023-07-04 | 2023-09-01 | 广东慧航天唯科技有限公司 | Used engine oil scheduling data safety management system based on data monitoring |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10659076B1 (en)* | 2019-04-30 | 2020-05-19 | EMC IP Holding Company LLC | Reducing the amount of data stored in a sequence of data blocks by combining deduplication and compression |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE60208012D1 (en)* | 2002-07-03 | 2006-01-19 | St Microelectronics Srl | Method, system and computer program for code compression |
| US8621182B1 (en)* | 2009-07-31 | 2013-12-31 | Amazon Technologies, Inc. | Management of object mapping information corresponding to a distributed storage system |
| JP5826114B2 (en)* | 2012-05-25 | 2015-12-02 | クラリオン株式会社 | Data decompression device, data compression device, data decompression program, data compression program, and compressed data distribution system |
| EP3015999A4 (en)* | 2013-09-29 | 2016-08-17 | Huawei Tech Co Ltd | Data processing method, system and client |
| US8836548B1 (en)* | 2013-12-05 | 2014-09-16 | Emc Corporation | Method and system for data compression at a storage system |
| US9569367B1 (en)* | 2014-03-04 | 2017-02-14 | EMC IP Holding Company LLC | Cache eviction based on types of data stored in storage systems |
| CN104579360B (en)* | 2015-02-04 | 2018-07-31 | 华为技术有限公司 | A kind of method and apparatus of data processing |
| CN106815287A (en)* | 2016-12-06 | 2017-06-09 | 中国银联股份有限公司 | A kind of buffer memory management method and device |
| US10162761B2 (en)* | 2017-03-21 | 2018-12-25 | Intel Corporation | Apparatus and method for system physical address to memory module address translation |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10659076B1 (en)* | 2019-04-30 | 2020-05-19 | EMC IP Holding Company LLC | Reducing the amount of data stored in a sequence of data blocks by combining deduplication and compression |
| Publication number | Publication date |
|---|---|
| CN112181869A (en) | 2021-01-05 |
| Publication | Publication Date | Title |
|---|---|---|
| CN112953550B (en) | Data compression method, electronic device and storage medium | |
| CN101783788B (en) | File compression method, file compression device, file decompression method, file decompression device, compressed file searching method and compressed file searching device | |
| CN112181869B (en) | Information storage method, device, server and medium | |
| US9998145B2 (en) | Data processing method and device | |
| CN112995199B (en) | Data encoding and decoding method, device, transmission system, terminal equipment and storage medium | |
| CN113300715B (en) | Data processing method, device, hardware compression equipment and medium | |
| CN101022552A (en) | Method and device for realizing Hoffman decodeng | |
| CN101534124B (en) | Compression algorithm for short natural language | |
| US11606103B2 (en) | Data compression method, data compression device, data decompression method, and data decompression device | |
| CN108880559B (en) | Data compression method, data decompression method, compression equipment and decompression equipment | |
| CN114095037B (en) | Application program updating method, updating data compression method, device and equipment | |
| CN110032432A (en) | The decompressing method and device of the compression method and device of example, example | |
| CN115189696A (en) | Hardware compression and decompression method based on Huffman decoding table | |
| CN116566397A (en) | Encoding method, decoding method, encoder, decoder, electronic device, and storage medium | |
| CN108829872A (en) | Immediate processing method, equipment, system and the storage medium of lossless compression file | |
| CN101741392A (en) | Huffman decoding method for fast resolving code length | |
| CN116318171B (en) | LZ4 decompression hardware acceleration realization/compression method, device, medium and chip | |
| CN119110343A (en) | Data compression, decompression method, device, equipment, storage device and vehicle | |
| CN114253479B (en) | CAN bus intrusion detection method and system | |
| CN115765756A (en) | Lossless data compression method, system and device for high-speed transparent transmission | |
| CN117200800A (en) | Data compression method, device, equipment and storage medium | |
| CN112486976A (en) | Data processing method, device, network equipment and storage medium | |
| CN118568569B (en) | Method, device, equipment and medium for processing long text based on classification model | |
| CN118796605A (en) | A Redis key memory usage monitoring method, device, equipment and storage medium | |
| CN119493781A (en) | Log information processing method, device and related equipment |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |