





技术领域technical field
本公开的实施例涉及计算机技术领域,具体涉及用于发送信息的方法及装置。The embodiments of the present disclosure relate to the field of computer technology, and in particular, to a method and an apparatus for sending information.
背景技术Background technique
日志文件是用于记录终端上系统操作事件的记录文件或文件集合,可分为事件日志和消息日志。具有处理历史数据、诊断问题的追踪以及理解系统的活动等重要作用。A log file is a record file or set of files used to record system operation events on a terminal, and can be divided into event logs and message logs. It plays an important role in processing historical data, diagnosing problem tracking, and understanding system activity.
服务器可以获取终端上的日志文件的日志数据,并通过日志数据对终端的运行状态、数据分析等数据处理。The server can obtain log data of log files on the terminal, and process data such as the running status and data analysis of the terminal through the log data.
发明内容SUMMARY OF THE INVENTION
本公开的实施例提出了用于发送信息的方法及装置。Embodiments of the present disclosure propose methods and apparatuses for transmitting information.
第一方面,本公开的实施例提供了一种用于发送信息的方法,该方法包括:响应于缓冲队列中存在待发送包,将上述缓冲队列中的待发送包封装成信息包,将上述信息包发送至信息服务器,其中,上述待发送包由信息采集协程发送至缓冲队列,上述信息采集协程用于从待处理日志文件中采集信息,并将采集的信息封装成待发送包;响应于接收到上述信息服务器发来的对应上述信息包的信息存储成功消息,向上述信息采集协程发送偏移标志更新指令,上述偏移标志更新指令用于指示上述信息采集协程采集待处理日志文件中信息的起点。In a first aspect, an embodiment of the present disclosure provides a method for sending information, the method comprising: in response to the existence of a to-be-sent packet in a buffer queue, encapsulating the to-be-sent packet in the buffer queue into an information packet, The information packet is sent to the information server, wherein the above-mentioned packet to be sent is sent to the buffer queue by the information collection coroutine, and the above-mentioned information collection coroutine is used to collect information from the log file to be processed, and encapsulate the collected information into a to-be-sent packet; In response to receiving the information storage success message corresponding to the above-mentioned information packet sent by the above-mentioned information server, send an offset flag update instruction to the above-mentioned information collection coroutine, and the above-mentioned offset flag update instruction is used to instruct the above-mentioned information collection coroutine to collect pending processing The starting point for information in the log file.
在一些实施例中,上述方法还包括:响应于检测到至少一个待处理日志文件,为上述至少一个待处理日志文件中的每个待处理日志文件设置信息采集协程和上述信息采集协程的信息采集周期。In some embodiments, the above method further includes: in response to detecting at least one log file to be processed, for each log file to be processed in the at least one log file to be processed, setting an information collection coroutine and an information collection coroutine of the above information collection coroutine. Information collection cycle.
在一些实施例中,上述待发送包的名称包括上述待处理日志文件的名称以及偏移标识的名称,上述偏移标识用于标记待处理日志文件中待读取信息的起点位置,以及,上述将上述缓冲队列中的待发送包封装成信息包,包括:根据上述待发送包的名称对上述缓冲队列中的待发送包进行划分,得到至少一个待发送包序列,其中,待发送包序列内的待发送包属于同一待处理日志文件,待发送包序列内的待发送包按照偏移标识的名称排序;对于上述至少一个待发送包序列中的待发送包序列,将该待发送包序列内的待发送包封装成信息包,并为信息包设置信息包名称。In some embodiments, the name of the above-mentioned packet to be sent includes the name of the above-mentioned log file to be processed and the name of an offset identifier, and the above-mentioned offset identifier is used to mark the starting position of the information to be read in the to-be-processed log file, and the above-mentioned Encapsulating the to-be-sent packets in the above-mentioned buffer queues into information packets includes: dividing the to-be-sent packets in the above-mentioned buffer queues according to the names of the above-mentioned to-be-sent packets to obtain at least one to-be-sent packet sequence, wherein, within the to-be-sent packet sequence The to-be-sent packets belong to the same to-be-processed log file, and the to-be-sent packets in the to-be-sent packet sequence are sorted according to the names of the offset identifiers; for the to-be-sent packet sequence in the at least one to-be-sent packet sequence, the to-be-sent packet sequence The packet to be sent is encapsulated into an information packet, and a packet name is set for the information packet.
在一些实施例中,上述将该待发送包序列内的待发送包封装成信息包,包括:按照顺序从上述待发送包序列中提取待发送包封装成信息包。In some embodiments, encapsulating the to-be-sent packets in the to-be-sent packet sequence into information packets includes: extracting to-be-sent packets from the to-be-sent packet sequence in sequence and encapsulating them into information packets.
在一些实施例中,上述为信息包设置信息包名称,包括:根据信息包内待发送包的偏移标识的名称为信息包设置信息包名称。In some embodiments, the above-mentioned setting the information package name for the information package includes: setting the information package name for the information package according to the name of the offset identifier of the to-be-sent packet in the information package.
在一些实施例中,上述将上述信息包发送至信息服务器,包括:响应于信息包的数据量大于数据量阈值,将上述信息包划分为不超过上述数据量阈值的至少两个子信息包;将上述至少两个子信息包中的子信息包发送至信息服务器。In some embodiments, the above-mentioned sending the above-mentioned information packet to the information server includes: in response to the data volume of the information packet being greater than the data volume threshold, dividing the above-mentioned information packet into at least two sub-information packets not exceeding the above-mentioned data volume threshold; A sub-packet of the at least two sub-packets is sent to the information server.
第二方面,本公开的实施例提供了一种用于发送信息的方法,该方法包括:获取待处理日志文件的偏移标识,上述偏移标识用于标记待处理日志文件中待读取信息的起点位置;将上述偏移标志指示的待读取信息作为信息起点,从上述待处理日志文件中获取待发送信息;将上述待发送信息封装成待发送包发送至缓冲队列。In a second aspect, an embodiment of the present disclosure provides a method for sending information, the method includes: acquiring an offset identifier of a log file to be processed, where the offset identifier is used to mark information to be read in the log file to be processed The starting position of the starting point; the information to be read indicated by the offset flag is used as the starting point of the information, and the information to be sent is obtained from the log file to be processed; the information to be sent is encapsulated into a packet to be sent and sent to the buffer queue.
在一些实施例中,上述获取待处理日志文件的偏移标识,包括:间隔设定的信息采集周期查询待处理日志文件;响应于上述待处理日志文件存在的更新信息,获取上述待处理日志文件的偏移标识。In some embodiments, obtaining the offset identifier of the log file to be processed includes: querying the log file to be processed at an information collection period set at intervals; obtaining the log file to be processed in response to the update information existing in the log file to be processed offset identifier.
在一些实施例中,上述获取上述待处理日志文件的偏移标识,包括:响应于上述更新信息的信息量大于设定信息量阈值,获取上述待处理日志文件的偏移标识。In some embodiments, obtaining the offset identifier of the log file to be processed includes: in response to the information amount of the update information being greater than a set information amount threshold, obtaining the offset identifier of the log file to be processed.
在一些实施例中,上述待处理日志文件内的信息设置有对应的时间信息,以及,上述将上述偏移标志指示的待读取信息作为信息起点,从上述待处理日志文件中获取待发送信息,包括:将上述偏移标志指示的待读取信息作为信息起点,将上述待处理日志文件中时间信息最后的信息的作为信息终点,将上述信息起点和信息终点之间的信息设置为待发送信息。In some embodiments, the information in the log file to be processed is set with corresponding time information, and the information to be read indicated by the offset flag is used as the starting point of the information, and the information to be sent is obtained from the log file to be processed. , including: taking the information to be read indicated by the above-mentioned offset flag as the starting point of the information, taking the last information of the time information in the above-mentioned log file to be processed as the ending point of the information, and setting the information between the starting point of the information and the ending point of the information as the information to be sent information.
在一些实施例中,上述将上述待发送信息封装成待发送包发送至缓冲队列,包括:将上述偏移标识和待发送信息封装成待发送包,并为上述待发送包设置名称,上述名称包括上述待处理日志文件的名称以及偏移标识的名称。In some embodiments, the above-mentioned encapsulating the above-mentioned to-be-sent information into a to-be-sent packet and sending it to a buffer queue includes: encapsulating the above-mentioned offset identifier and the to-be-sent information into a to-be-sent packet, and setting a name for the above-mentioned to-be-sent packet, the above name Include the name of the above pending log file and the name of the offset identifier.
在一些实施例中,上述方法还包括:响应于接收到偏移标志更新指令,为上述待处理日志文件中的、对应上述待发送信息的最后一条信息设置更新偏移标志,上述更新偏移标志用于标记待处理日志文件中下一次待读取信息的起点位置。In some embodiments, the above method further includes: in response to receiving the offset flag update instruction, setting an update offset flag for the last piece of information in the above log file to be processed and corresponding to the above to-be-sent information, the above-mentioned update offset flag It is used to mark the starting position of the next information to be read in the log file to be processed.
第三方面,本公开的实施例提供了一种用于发送信息的装置,该装置包括:第一信息发送单元,响应于缓冲队列中存在待发送包,将上述缓冲队列中的待发送包封装成信息包,被配置成将上述信息包发送至信息服务器,其中,上述待发送包由信息采集协程发送至缓冲队列,上述信息采集协程用于从待处理日志文件中采集信息,并将采集的信息封装成待发送包;第二信息发送单元,响应于接收到上述信息服务器发来的对应上述信息包的信息存储成功消息,被配置成向上述信息采集协程发送偏移标志更新指令,上述偏移标志更新指令用于指示上述信息采集协程采集待处理日志文件中信息的起点。In a third aspect, an embodiment of the present disclosure provides an apparatus for sending information, the apparatus includes: a first information sending unit that, in response to the presence of a to-be-sent packet in the buffer queue, encapsulates the to-be-sent packet in the buffer queue It is configured to send the above-mentioned information packet to the information server, wherein, the above-mentioned to-be-sent packet is sent to the buffer queue by the information collection coroutine, and the above-mentioned information collection coroutine is used to collect information from the log file to be processed, and to The collected information is encapsulated into a packet to be sent; the second information sending unit, in response to receiving the information storage success message corresponding to the information packet sent by the information server, is configured to send an offset flag update instruction to the information collection coroutine , the above offset flag update instruction is used to instruct the above information collection coroutine to collect the starting point of the information in the log file to be processed.
在一些实施例中,上述装置还包括:信息采集协程设置单元,响应于检测到至少一个待处理日志文件,被配置成为上述至少一个待处理日志文件中的每个待处理日志文件设置信息采集协程和上述信息采集协程的信息采集周期。In some embodiments, the above apparatus further includes: an information collection coroutine setting unit, configured to set information collection for each to-be-processed log file in the at least one to-be-processed log file in response to detecting at least one to-be-processed log file The information collection period of the coroutine and the above information collection coroutine.
在一些实施例中,上述待发送包的名称包括上述待处理日志文件的名称以及偏移标识的名称,上述偏移标识用于标记待处理日志文件中待读取信息的起点位置,以及,上述第一信息发送单元包括:序列划分子单元,被配置成根据上述待发送包的名称对上述缓冲队列中的待发送包进行划分,得到至少一个待发送包序列,其中,待发送包序列内的待发送包属于同一待处理日志文件,待发送包序列内的待发送包按照偏移标识的名称排序;信息包封装子单元,对于上述至少一个待发送包序列中的待发送包序列,被配置成将该待发送包序列内的待发送包封装成信息包,并为信息包设置信息包名称。In some embodiments, the name of the above-mentioned packet to be sent includes the name of the above-mentioned log file to be processed and the name of an offset identifier, and the above-mentioned offset identifier is used to mark the starting position of the information to be read in the to-be-processed log file, and the above-mentioned The first information sending unit includes: a sequence dividing subunit, configured to divide the to-be-sent packets in the above-mentioned buffer queue according to the names of the above-mentioned to-be-sent packets to obtain at least one to-be-sent packet sequence, wherein the The packets to be sent belong to the same log file to be processed, and the packets to be sent in the sequence of packets to be sent are sorted according to the names of the offset identifiers; the information packet encapsulation subunit is configured for the sequence of packets to be sent in the at least one sequence of packets to be sent. to encapsulate the to-be-sent packets in the to-be-sent packet sequence into information packets, and set an information packet name for the information packets.
在一些实施例中,上述信息包封装子单元包括:信息包封装模块,被配置成按照顺序从上述待发送包序列中提取待发送包封装成信息包。In some embodiments, the above-mentioned information packet encapsulation subunit includes: an information packet encapsulation module configured to sequentially extract the to-be-sent packets from the above-mentioned to-be-sent packet sequence and encapsulate them into information packets.
在一些实施例中,上述信息包封装子单元包括:信息包名称设置模块,被配置成根据信息包内待发送包的偏移标识的名称为信息包设置信息包名称。In some embodiments, the above-mentioned information packet encapsulation subunit includes: an information packet name setting module configured to set an information packet name for the information packet according to the name of the offset identifier of the to-be-sent packet in the information packet.
在一些实施例中,上述第一信息发送单元包括:信息包划分子单元,响应于信息包的数据量大于数据量阈值,被配置成将上述信息包划分为不超过上述数据量阈值的至少两个子信息包;第一信息发送子单元,被配置成将上述至少两个子信息包中的子信息包发送至信息服务器。In some embodiments, the above-mentioned first information sending unit includes: an information packet dividing subunit, configured to divide the above-mentioned information packet into at least two parts that do not exceed the above-mentioned data amount threshold in response to the data amount of the information packet being greater than the data amount threshold value The first information sending subunit is configured to send a sub-information packet of the at least two sub-information packets to the information server.
第四方面,本公开的实施例提供了一种用于发送信息的装置,该装置包括:偏移标识获取单元,被配置成获取待处理日志文件的偏移标识,上述偏移标识用于标记待处理日志文件中待读取信息的起点位置;待发送信息获取单元,被配置成将上述偏移标志指示的待读取信息作为信息起点,从上述待处理日志文件中获取待发送信息;第三信息发送单元,被配置成将上述待发送信息封装成待发送包发送至缓冲队列。In a fourth aspect, an embodiment of the present disclosure provides an apparatus for sending information, the apparatus includes: an offset identifier acquiring unit configured to acquire an offset identifier of a log file to be processed, where the offset identifier is used for marking The starting point position of the information to be read in the log file to be processed; the information acquisition unit to be sent is configured to use the information to be read indicated by the offset flag as the starting point of the information, and obtain the information to be sent from the log file to be processed; The third information sending unit is configured to encapsulate the above-mentioned information to be sent into a to-be-sent packet and send it to the buffer queue.
在一些实施例中,上述偏移标识获取单元包括:日志查询子单元,被配置成间隔设定的信息采集周期查询待处理日志文件;偏移标识获取子单元,响应于上述待处理日志文件存在的更新信息,被配置成获取上述待处理日志文件的偏移标识。In some embodiments, the above offset identifier acquisition unit includes: a log query subunit, configured to query the log files to be processed at an information collection period set at intervals; the offset identifier acquisition subunit, in response to the existence of the above log files to be processed The update information is configured to obtain the offset identifier of the log file to be processed.
在一些实施例中,上述偏移标识获取子单元包括:偏移标识获取模块,响应于上述更新信息的信息量大于设定信息量阈值,被配置成获取上述待处理日志文件的偏移标识。In some embodiments, the offset identifier acquisition subunit includes: an offset identifier acquisition module configured to acquire the offset identifier of the log file to be processed in response to the information amount of the update information being greater than a set information amount threshold.
在一些实施例中,上述待处理日志文件内的信息设置有对应的时间信息,以及,上述待发送信息获取单元包括:待发送信息获取子单元,被配置成将上述偏移标志指示的待读取信息作为信息起点,将上述待处理日志文件中时间信息最后的信息的作为信息终点,将上述信息起点和信息终点之间的信息设置为待发送信息。In some embodiments, the information in the log file to be processed is set with corresponding time information, and the information to-be-sent acquisition unit includes: a to-be-sent information acquisition subunit configured to The information is taken as the starting point of the information, the last information of the time information in the log file to be processed is used as the ending point of the information, and the information between the starting point and the ending point of the information is set as the information to be sent.
在一些实施例中,上述第三信息发送单元包括:第三信息发送子单元,被配置成将上述偏移标识和待发送信息封装成待发送包,并为上述待发送包设置名称,上述名称包括上述待处理日志文件的名称以及偏移标识的名称。In some embodiments, the third information sending unit includes: a third information sending subunit, configured to encapsulate the offset identifier and the to-be-sent information into a to-be-sent packet, and to set a name for the to-be-sent packet, and the above name Include the name of the above pending log file and the name of the offset identifier.
在一些实施例中,上述装置还包括:偏移标志更新单元,响应于接收到偏移标志更新指令,被配置成为上述待处理日志文件中的、对应上述待发送信息的最后一条信息设置更新偏移标志,上述更新偏移标志用于标记待处理日志文件中下一次待读取信息的起点位置。In some embodiments, the above-mentioned apparatus further includes: an offset flag update unit, configured to set an update offset for the last piece of information in the above-mentioned log file to be processed and corresponding to the above-mentioned to-be-sent information in response to receiving the offset flag update instruction The shift flag, the above-mentioned update offset flag is used to mark the starting position of the information to be read next time in the log file to be processed.
第五方面,本公开的实施例提供了一种电子设备,包括:一个或多个处理器;存储器,其上存储有一个或多个程序,当上述一个或多个程序被上述一个或多个处理器执行时,使得上述一个或多个处理器执行上述第一方面的用于发送信息的方法或第二方面的用于发送信息的方法。In a fifth aspect, embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a memory on which one or more programs are stored, when the one or more programs are stored by the one or more programs described above When executed by the processor, the above one or more processors are caused to execute the method for sending information of the first aspect or the method for sending information of the second aspect.
第六方面,本公开的实施例提供了一种计算机可读介质,其上存储有计算机程序,其特征在于,该程序被处理器执行时实现上述第一方面的用于发送信息的方法或第二方面的用于发送信息的方法。In a sixth aspect, embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the method for sending information of the first aspect or the first aspect is implemented. A method for sending information in two aspects.
本公开的实施例提供的用于发送信息的方法及装置,首先在缓冲队列中存在待发送包时,将上述缓冲队列中的待发送包封装成信息包,将上述信息包发送至信息服务器,其中,上述待发送包由信息采集协程发送至缓冲队列,上述信息采集协程用于从待处理日志文件中采集信息,并将采集的信息封装成待发送包;然后在接收到上述信息服务器发来的对应上述信息包的信息存储成功消息时,向上述信息采集协程发送偏移标志更新指令,上述偏移标志更新指令用于指示上述信息采集协程采集待处理日志文件中信息的起点。本申请能够同时获取多个待处理日志文件的信息,实现了对日志信息的并行采集,提高了日志信息的采集效率。In the method and device for sending information provided by the embodiments of the present disclosure, first, when there are packets to be sent in the buffer queue, the packets to be sent in the buffer queue are encapsulated into information packets, and the information packets are sent to the information server, The above-mentioned packets to be sent are sent to the buffer queue by the information collection coroutine, and the above information collection coroutine is used to collect information from the log files to be processed, and encapsulate the collected information into packets to be sent; When the information storage success message corresponding to the above-mentioned information packet is sent, an offset flag update instruction is sent to the above-mentioned information collection coroutine, and the above-mentioned offset flag update instruction is used to instruct the above-mentioned information collection coroutine to collect the starting point of the information in the log file to be processed. . The present application can simultaneously acquire information of multiple log files to be processed, realizes parallel collection of log information, and improves the collection efficiency of log information.
附图说明Description of drawings
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述,本公开的其它特征、目的和优点将会变得更明显:Other features, objects and advantages of the present disclosure will become more apparent upon reading the detailed description of non-limiting embodiments taken with reference to the following drawings:
图1是本公开的一个实施例可以应用于其中的示例性系统架构图;FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure may be applied;
图2是根据本公开的用于发送信息的方法的一个实施例的流程图;Figure 2 is a flow diagram of one embodiment of a method for sending information according to the present disclosure;
图3是根据本公开的用于发送信息的方法的一个应用场景的示意图;3 is a schematic diagram of an application scenario of the method for sending information according to the present disclosure;
图4是根据本公开的用于发送信息的方法的另一个实施例的流程图;4 is a flowchart of another embodiment of a method for sending information according to the present disclosure;
图5是根据本公开的用于发送信息的装置的一个实施例的结构示意图;5 is a schematic structural diagram of an embodiment of an apparatus for sending information according to the present disclosure;
图6是根据本公开的用于发送信息的装置的另一个实施例的结构示意图;6 is a schematic structural diagram of another embodiment of an apparatus for sending information according to the present disclosure;
图7是适于用来实现本公开的实施例的电子设备结构示意图。FIG. 7 is a schematic structural diagram of an electronic device suitable for implementing embodiments of the present disclosure.
具体实施方式Detailed ways
下面结合附图和实施例对本公开作进一步的详细说明。可以理解的是,此处所描述的具体实施例仅仅用于解释相关发明,而非对该发明的限定。另外还需要说明的是,为了便于描述,附图中仅示出了与有关发明相关的部分。The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. In addition, it should be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.
需要说明的是,在不冲突的情况下,本公开中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本公开。It should be noted that the embodiments of the present disclosure and the features of the embodiments may be combined with each other under the condition of no conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
图1示出了可以应用本公开的实施例的用于发送信息的方法或用于发送信息的装置的示例性系统架构100。FIG. 1 illustrates an
如图1所示,系统架构100可以包括终端设备101、102、103,网络104、日志采集设备105和信息服务器106。网络104用以在终端设备101、102、103、日志采集设备105和信息服务器106之间提供通信链路的介质。网络104可以包括各种连接类型,例如有线、无线通信链路或者光纤电缆等等。As shown in FIG. 1 , the
终端设备101、102、103通过网络104与日志采集设备105交互,以接收或发送消息等。终端设备101、102、103上可以安装有各种通讯客户端应用,例如网页浏览器应用、购物类应用、搜索类应用、即时通信工具、邮箱客户端、社交平台软件等。The
终端设备101、102、103可以是硬件,也可以是软件。当终端设备101、102、103为硬件时,可以是具有显示屏并且支持数据处理的各种电子设备,包括但不限于智能手机、平板电脑、电子书阅读器、MP3播放器(Moving Picture Experts Group Audio Layer III,动态影像专家压缩标准音频层面3)、MP4(Moving Picture Experts Group Audio Layer IV,动态影像专家压缩标准音频层面4)播放器、膝上型便携计算机和台式计算机等等。当终端设备101、102、103为软件时,可以安装在上述所列举的电子设备中。其可以实现成多个软件或软件模块(例如用来提供分布式服务),也可以实现成单个软件或软件模块,在此不做具体限定。The
日志采集设备105可以用于对终端设备101、102、103上的日志文件设置信息采集协程,以通过信息采集协程对日志文件进行数据采集。日志采集设备105可以将接收到的日志数据发送给信息服务器106。The
需要说明的是,日志采集设备105可以是硬件,也可以是软件。当日志采集设备105为硬件时,可以实现成多个服务器组成的分布式服务器集群,也可以实现成单个服务器。当日志采集设备105为软件时,可以实现成多个软件或软件模块(例如用来提供分布式服务),也可以实现成单个软件或软件模块,在此不做具体限定。It should be noted that the
信息服务器106可以是提供各种服务的服务器,例如对日志采集设备105发来的日志数据进行处理的服务器。服务器可以对接收到的日志数据等数据进行分析等处理,并将处理结果保存至服务器。The
需要说明的是,本公开的实施例所提供的用于发送信息的方法一般由日志采集设备105执行,相应地,用于发送信息的装置一般设置于日志采集设备105中。It should be noted that the method for sending information provided by the embodiments of the present disclosure is generally performed by the
需要说明的是,服务器可以是硬件,也可以是软件。当服务器为硬件时,可以实现成多个服务器组成的分布式服务器集群,也可以实现成单个服务器。当服务器为软件时,可以实现成多个软件或软件模块(例如用来提供分布式服务),也可以实现成单个软件或软件模块,在此不做具体限定。It should be noted that the server may be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or can be implemented as a single server. When the server is software, it may be implemented as multiple software or software modules (for example, used to provide distributed services), or may be implemented as a single software or software module, which is not specifically limited herein.
应该理解,图1中的终端设备、网络和服务器的数目仅仅是示意性的。根据实现需要,可以具有任意数目的终端设备、网络和服务器。It should be understood that the numbers of terminal devices, networks and servers in FIG. 1 are merely illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.
继续参考图2,示出了根据本公开的用于发送信息的方法的一个实施例的流程200。该用于发送信息的方法包括以下步骤:With continued reference to FIG. 2, a
步骤201,响应于缓冲队列中存在待发送包,将上述缓冲队列中的待发送包封装成信息包,将上述信息包发送至信息服务器。
在本实施例中,用于发送信息的方法的执行主体(例如图1所示的日志采集设备105)可以通过有线连接方式或者无线连接方式接收终端设备101、102、103上信息采集协程发送至缓冲序列的待发送包,其中,缓冲序列位于执行主体上,用于存放待发送包。待发送包包含了信息采集协程采集的待处理日志文件中的信息。需要指出的是,上述无线连接方式可以包括但不限于3G/4G连接、WiFi连接、蓝牙连接、WiMAX连接、Zigbee连接、UWB(UltraWideband)连接、以及其他现在已知或将来开发的无线连接方式。In this embodiment, the execution body of the method for sending information (for example, the
现有的日志信息采集方法中,虽然可以实现对日志信息的周期性采集,但只能按行读取并处理,无法高效地将日志信息推送至信息服务器106。此外,现有的日志信息采集方法在采集日志信息的过程中,由于时间间隔等原因,会产生大量的小文件,且不能对同一设备上的多个日志文件进行并行采集。In the existing log information collection method, although periodic collection of log information can be realized, it can only be read and processed by line, and the log information cannot be efficiently pushed to the
本申请在执行主体上的缓冲队列中存在待发送包时,将缓冲队列中的待发送包封装成信息包,将信息包发送至信息服务器106。执行主体并不是在接收到每一个待发送包时就将待发送包发送给信息服务器,而是需要将待发送包封装成信息包后才要发送给信息服务器。封装过程可以有效地减少发送待发送包的次数和数量,有效降低了出现小文件的情况。其中,上述待发送包由信息采集协程发送至缓冲队列。上述信息采集协程可以用于从待处理日志文件中采集信息,并将采集的信息封装成待发送包。信息采集协程可以是待处理日志文件所在的设备上运行的一个线程或进程,具体根据实际需要而设定。In the present application, when there is a packet to be sent in the buffer queue on the execution body, the packet to be sent in the buffer queue is encapsulated into an information packet, and the information packet is sent to the
在本实施例的一些可选的实现方式中,上述方法还可以包括:响应于检测到至少一个待处理日志文件,为上述至少一个待处理日志文件中的每个待处理日志文件设置信息采集协程和上述信息采集协程的信息采集周期。In some optional implementations of this embodiment, the above method may further include: in response to detecting at least one log file to be processed, setting an information collection protocol for each log file to be processed in the at least one log file to be processed process and the information collection cycle of the above-mentioned information collection coroutine.
执行主体可以与多个终端设备101、102、103同时建立信息连接,并监测终端设备101、102、103上的信息变化。当执行主体监测到终端设备101、102、103上存在新的至少一个待处理日志文件时,可以分别为至少一个待处理日志文件中的每个待处理日志文件设置信息采集协程。例如,执行主体可以向终端设备101、102、103发送指令,指示终端设备101、102、103分配一个线程或进程,用于采集对应待处理日志文件的日志信息。终端设备101、102、103接收到该指令后,为待处理日志文件分配一个专用于采集信息的线程或进程。如此,实现了对多个日志文件的并行信息采集。之后,执行主体还可以进一步设置信息采集协程的信息采集周期,以使得信息服务器106能够较为平稳地获取日志信息,避免出现短时大量日志信息的传输造成网络压力,并减少小文件的产生。The execution body can establish information connections with multiple
在本实施例的一些可选的实现方式中,上述待发送包的名称可以包括上述待处理日志文件的名称以及偏移标识的名称,上述偏移标识用于标记待处理日志文件中待读取信息的起点位置,以及,上述将上述缓冲队列中的待发送包封装成信息包,可以包括以下步骤:In some optional implementations of this embodiment, the name of the packet to be sent may include the name of the log file to be processed and the name of an offset identifier, where the offset identifier is used to mark the log file to be read in the to-be-processed log file The starting point position of the information, and the above-mentioned encapsulation of the to-be-sent packets in the above-mentioned buffer queue into information packets may include the following steps:
第一步,根据上述待发送包的名称对上述缓冲队列中的待发送包进行划分,得到至少一个待发送包序列。The first step is to divide the to-be-sent packets in the above-mentioned buffer queue according to the names of the above-mentioned to-be-sent packets to obtain at least one to-be-sent packet sequence.
信息采集协程为了对待发送包进行标记,可以对待发送包的名称设置名称。待发送包的名称可以包括上述待处理日志文件的名称以及偏移标识的名称。偏移标识用于标记信息采集协程采集待处理日志文件中待读取信息的起点位置。偏移标识的名称可以为待处理日志文件的行号等,用于表征信息在待处理日志文件的先后顺序。由上述描述可知,执行主体可以并行获取多个待处理日志文件的待发送包。为了避免信息传输过程中出现信息混乱,执行主体还可以根据上述待发送包的名称对上述缓冲队列中的待发送包进行划分,得到至少一个待发送包序列。其中,待发送包序列内的待发送包属于同一待处理日志文件。待发送包序列内的待发送包可以按照偏移标识的名称排序。In order to mark the packet to be sent, the information collection coroutine can set a name for the name of the packet to be sent. The name of the package to be sent may include the name of the log file to be processed and the name of the offset identifier. The offset identifier is used to mark the starting point for the information collection coroutine to collect the information to be read in the log file to be processed. The name of the offset identifier may be the line number of the log file to be processed, etc., and is used to represent the sequence of information in the log file to be processed. It can be seen from the above description that the execution body can acquire the to-be-sent packets of multiple to-be-processed log files in parallel. In order to avoid information confusion during the information transmission process, the execution body may further divide the to-be-sent packets in the buffer queue according to the names of the to-be-sent packets to obtain at least one to-be-sent packet sequence. The to-be-sent packets in the to-be-sent packet sequence belong to the same to-be-processed log file. The to-be-sent packets in the to-be-sent packet sequence may be sorted according to the names of the offset identifiers.
由于偏移标识的名称可以表征信息在待处理日志文件中的先后顺序。因此,执行主体可以按照偏移标识的名称对待发送包序列内的待发送包排序。又由于信息采集协程是按照偏移标识在待处理日志文件采集信息并发送待发送包的。即,每个待发送包之间具有连贯的先后顺序。因此,执行主体得到的待发送包序列内的待发送包也是连续的。The name of the offset identifier can represent the sequence of information in the log file to be processed. Therefore, the execution body can sort the to-be-sent packets in the to-be-sent packet sequence according to the names identified by the offsets. In addition, because the information collection coroutine collects information in the log file to be processed according to the offset identifier and sends the to-be-sent packet. That is, each packet to be sent has a continuous sequence. Therefore, the to-be-sent packets in the to-be-sent packet sequence obtained by the execution subject are also continuous.
第二步,对于上述至少一个待发送包序列中的待发送包序列,将该待发送包序列内的待发送包封装成信息包,并为信息包设置信息包名称。In the second step, for the to-be-sent packet sequence in the at least one to-be-sent packet sequence, encapsulate the to-be-sent packets in the to-be-sent packet sequence into information packets, and set an information packet name for the information packets.
执行主体可以对每个待发送包序列内的待发送包进行封装,以避免不同待处理日志文件的待发送包封装在一起。如此,也便于为信息包设置信息包名称。The execution body may encapsulate the to-be-sent packets in each to-be-sent packet sequence, so as to avoid encapsulating the to-be-sent packets of different to-be-processed log files together. In this way, it is also convenient to set the packet name for the packet.
在本实施例的一些可选的实现方式中,上述将该待发送包序列内的待发送包封装成信息包,可以包括:按照顺序从上述待发送包序列中提取待发送包封装成信息包。In some optional implementation manners of this embodiment, the above-mentioned encapsulating the to-be-sent packets in the to-be-sent packet sequence into information packets may include: extracting the to-be-sent packets from the above-mentioned to-be-sent packet sequence in order and encapsulating them into information packets .
为了避免出现信息混乱,执行主体可以进一步按照顺序从上述待发送包序列中提取待发送包封装成信息包。In order to avoid information confusion, the execution body may further extract the to-be-sent packets from the above-mentioned sequence of to-be-sent packets in sequence and encapsulate them into information packets.
在本实施例的一些可选的实现方式中,上述为信息包设置信息包名称,可以包括:根据信息包内待发送包的偏移标识的名称为信息包设置信息包名称。In some optional implementations of this embodiment, the above-mentioned setting of the information package name for the information package may include: setting the information package name for the information package according to the name of the offset identifier of the to-be-sent packet in the information package.
执行主体还可以根据信息包内待发送包的偏移标识的名称为信息包设置信息包名称。例如,信息包内包含三个待发送包。三个待发送包的名称可以分别是:文件1-第50行;文件1-第150行;文件1-第300行。其中,“文件1”为待处理日志文件的名称;“第50行”、“第150行”、“第300行”可以为偏移标识的名称。则信息包名称可以为:文件1-第50-300行。The execution body may also set an information packet name for the information packet according to the name of the offset identifier of the to-be-sent packet in the information packet. For example, the packet contains three packets to be sent. The names of the three packages to be sent may be: file 1-line 50; file 1-line 150; file 1-line 300. Among them, "File 1" is the name of the log file to be processed; "Line 50", "Line 150", and "Line 300" can be the names of the offset identifiers. Then the package name can be: file 1-line 50-300.
在本实施例的一些可选的实现方式中,上述将上述信息包发送至信息服务器,可以包括以下步骤:In some optional implementations of this embodiment, the above-mentioned sending the above-mentioned information packet to the information server may include the following steps:
第一步,响应于信息包的数据量大于数据量阈值,将上述信息包划分为不超过上述数据量阈值的至少两个子信息包。In the first step, in response to the data volume of the information packet being greater than the data volume threshold, the above-mentioned information packet is divided into at least two sub-information packets which do not exceed the above-mentioned data volume threshold.
为保证每次向信息服务器发送信息包的成功率,执行主体可以按照数据量阈值对信息包的数据量进行限制。当信息包的数据量大于数据量阈值时,执行主体可以将上述信息包划分为不超过数据量阈值的至少两个子信息包。例如,信息包名称可以为:文件1-第50-300行,划分得到两个子信息包的名称可以为:“文件1-第50-300行-包1”和“文件1-第50-300行-包2”。In order to ensure the success rate of sending an information packet to the information server each time, the execution subject may limit the data volume of the information packet according to the data volume threshold. When the data volume of the information packet is greater than the data volume threshold, the execution body may divide the above-mentioned information packet into at least two sub-information packets that do not exceed the data volume threshold. For example, the package name can be: file 1-line 50-300, and the names of two sub-packages can be divided into: "file 1-line 50-300-package 1" and "file 1-line 50-300" Line-Package 2".
第二步,将上述至少两个子信息包中的子信息包发送至信息服务器。In the second step, the sub-information package of the at least two sub-information packages mentioned above is sent to the information server.
之后,执行主体可以将子信息包按照划分后的顺序发送至信息服务器。Afterwards, the execution body may send the sub-information packets to the information server in the divided order.
步骤202,响应于接收到上述信息服务器发来的对应上述信息包的信息存储成功消息,向上述信息采集协程发送偏移标志更新指令。
当信息服务器接收到信息包并存储后,可以向执行主体发送信息存储成功消息。即信息存储成功消息用于表征信息包已经被信息服务器成功存储。执行主体接收到信息存储成功消息后,可以向信息采集协程发送偏移标志更新指令。信息采集协程接收到偏移标志更新指令后,可以修改采集待处理日志文件中信息的位置,以避免重复采集信息。即,上述偏移标志更新指令可以用于指示上述信息采集协程采集待处理日志文件中信息的起点。After the information server receives and stores the information packet, it can send the information storage success message to the execution body. That is, the information storage success message is used to indicate that the information packet has been successfully stored by the information server. After receiving the information storage success message, the execution body can send an offset flag update instruction to the information collection coroutine. After the information collection coroutine receives the offset flag update instruction, it can modify the location where information in the log file to be processed is collected to avoid repeated collection of information. That is, the above-mentioned offset flag update instruction may be used to instruct the above-mentioned information collection coroutine to collect the starting point of the information in the log file to be processed.
继续参见图3,图3是根据本实施例的用于发送信息的方法的应用场景的一个示意图。在图3的应用场景中,日志采集设备105的缓冲队列中存在信息采集协程1采集的终端设备102的待发送包、信息采集协程2采集的终端设备103的待发送包。日志采集设备105可以将待发送包封装成信息包,之后将信息包发送至信息服务器106。当日志采集设备105接收到信息服务器106发来的信息存储成功消息时,说明日志采集设备105发送的信息包已经被信息服务器106成功存储。日志采集设备105可以向信息采集协程1和信息采集协程2发送偏移标志更新指令,指示信息采集协程1和信息采集协程2调整终端设备102和终端设备103上待处理日志文件的下一次采集起点。Continue to refer to FIG. 3 , which is a schematic diagram of an application scenario of the method for sending information according to this embodiment. In the application scenario of FIG. 3 , in the buffer queue of the
本公开的上述实施例提供的方法首先在缓冲队列中存在待发送包时,将上述缓冲队列中的待发送包封装成信息包,将上述信息包发送至信息服务器;然后在接收到上述信息服务器发来的对应上述信息包的信息存储成功消息时,向上述信息采集协程发送偏移标志更新指令。本申请能够同时获取多个待处理日志文件的信息,实现了对日志信息的并行采集,提高了日志信息的采集效率。The method provided by the above embodiments of the present disclosure firstly encapsulates the to-be-sent packets in the buffer queue into information packets when there are to-be-sent packets in the buffer queue, and sends the above-mentioned information packets to the information server; then, after receiving the above-mentioned information server When the information storage success message corresponding to the above-mentioned information packet is sent, an offset flag update instruction is sent to the above-mentioned information collection coroutine. The present application can simultaneously acquire information of multiple log files to be processed, realizes parallel collection of log information, and improves the collection efficiency of log information.
进一步参考图4,其示出了用于发送信息的方法的又一个实施例的流程400。该用于发送信息的方法的流程400,包括以下步骤:With further reference to Figure 4, a
步骤401,获取待处理日志文件的偏移标识。Step 401: Obtain the offset identifier of the log file to be processed.
在本实施例中,用于发送信息的方法的执行主体(例如图1所示的终端设备101、102、103)在采集待处理日志文件的信息之前,可以首先通过信息采集协程获取待处理日志文件的偏移标识。其中,上述偏移标识用于标记待处理日志文件中待读取信息的起点位置。In this embodiment, before collecting the information of the log file to be processed, the execution subject of the method for sending information (for example, the
在本实施例的一些可选的实现方式中,上述获取待处理日志文件的偏移标识,可以包括以下步骤:In some optional implementations of this embodiment, obtaining the offset identifier of the log file to be processed above may include the following steps:
第一步,间隔设定的信息采集周期查询待处理日志文件。The first step is to query the pending log files at the set information collection cycle.
待处理日志文件内信息的变化情况不定。为了及时采集待处理日志文件内的信息,避免待处理日志文件出现大量新的信息后才采集信息,导致信息服务器106不能及时获取到日志信息。执行主体可以间隔设定的信息采集周期查询待处理日志文件。Changes to the information in the pending log file are uncertain. In order to collect the information in the log file to be processed in time, it is avoided that the
第二步,响应于上述待处理日志文件存在的更新信息,获取上述待处理日志文件的偏移标识。In the second step, in response to the update information existing in the log file to be processed, the offset identifier of the log file to be processed is acquired.
当待处理日志文件存在的更新信息(即未被信息采集协程采集过的信息)时,执行主体可以获取上述待处理日志文件的偏移标识。When there is updated information in the log file to be processed (that is, information that has not been collected by the information collection protocol), the execution subject can obtain the offset identifier of the log file to be processed.
在本实施例的一些可选的实现方式中,上述获取上述待处理日志文件的偏移标识,可以包括:响应于上述更新信息的信息量大于设定信息量阈值,获取上述待处理日志文件的偏移标识。In some optional implementations of this embodiment, obtaining the offset identifier of the log file to be processed may include: in response to the information amount of the update information being greater than a set information amount threshold, obtaining the log file to be processed. Offset ID.
如果更新信息的信息量较少,执行主体可以不对其进行更新,以避免过多地发送信息量很少的待发送包。当更新信息的信息量大于设定信息量阈值时,执行主体可以获取上述待处理日志文件的偏移标识,以采集待处理日志文件内的信息。If the update information has a small amount of information, the execution body may not update it, so as to avoid sending too many packets to be sent with a small amount of information. When the information amount of the update information is greater than the set information amount threshold, the execution body may obtain the offset identifier of the log file to be processed, so as to collect the information in the log file to be processed.
步骤402,将上述偏移标志指示的待读取信息作为信息起点,从上述待处理日志文件中获取待发送信息。
偏移标识用于标记待处理日志文件中待读取信息的起点位置,执行主体可以将偏移标志指示的待读取信息作为信息起点,从待处理日志文件中获取待发送信息。The offset flag is used to mark the starting point of the information to be read in the log file to be processed, and the execution body can use the information to be read indicated by the offset flag as the starting point of the information to obtain the information to be sent from the log file to be processed.
在本实施例的一些可选的实现方式中,上述待处理日志文件内的信息设置有对应的时间信息,以及,上述将上述偏移标志指示的待读取信息作为信息起点,从上述待处理日志文件中获取待发送信息,可以包括:将上述偏移标志指示的待读取信息作为信息起点,将上述待处理日志文件中时间信息最后的信息的作为信息终点,将上述信息起点和信息终点之间的信息设置为待发送信息。In some optional implementations of this embodiment, the information in the log file to be processed is set with corresponding time information, and the information to be read indicated by the offset flag is used as the starting point of the information, and from the to-be-processed information Obtaining the information to be sent from the log file may include: taking the information to be read indicated by the offset flag as the starting point of the information, taking the last information of the time information in the log file to be processed as the ending point of the information, and using the starting point of the information and the ending point of the information The information between is set as the information to be sent.
待处理日志文件内的信息通常都设置有对应的时间信息,以标记该信息的发生时间。为此,执行主体可以将偏移标志指示的待读取信息作为信息起点,将待处理日志文件中时间信息最后的信息的作为信息终点,将上述信息起点和信息终点之间的信息设置为待发送信息。The information in the log file to be processed is usually set with corresponding time information to mark the occurrence time of the information. To this end, the execution body can use the information to be read indicated by the offset flag as the starting point of the information, use the last information of the time information in the log file to be processed as the ending point of the information, and set the information between the starting point of the information and the ending point of the information as the starting point of the information to be processed. send Message.
步骤403,将上述待发送信息封装成待发送包发送至终端的缓冲队列。Step 403: Encapsulate the above information to be sent into a packet to be sent and send it to the buffer queue of the terminal.
读取到待读取信息后,执行主体可以将待发送信息封装成待发送包发送至日志采集设备105的缓冲队列。After reading the to-be-read information, the execution body may encapsulate the to-be-sent information into to-be-sent packets and send them to the buffer queue of the
在本实施例的一些可选的实现方式中,上述将上述待发送信息封装成待发送包发送至缓冲队列,可以包括:将上述偏移标识和待发送信息封装成待发送包,并为上述待发送包设置名称。In some optional implementations of this embodiment, the above-mentioned encapsulating the above-mentioned information to be sent into a to-be-sent packet and sending it to the buffer queue may include: encapsulating the above-mentioned offset identifier and the to-be-sent information into a to-be-sent packet, which is the above-mentioned Set the name of the package to be sent.
为了便于信息服务器106对日志信息的归档等操作,执行主体可以将上述偏移标识和待发送信息封装成待发送包,并为上述待发送包设置名称。上述名称可以包括上述待处理日志文件的名称以及偏移标识的名称。In order to facilitate operations such as archiving of log information by the
在本实施例的一些可选的实现方式中,上述方法还可以包括:响应于接收到偏移标志更新指令,为上述待处理日志文件中的、对应上述待发送信息的最后一条信息设置更新偏移标志。In some optional implementations of this embodiment, the above method may further include: in response to receiving an offset flag update instruction, setting an update offset for the last piece of information in the above log file to be processed and corresponding to the above information to be sent Move the sign.
当信息服务器106接收到对应的信息后,会向日志采集设备105发送信息存储成功消息。之后,日志采集设备105会向执行主体发送偏移标志更新指令。执行主体接收到偏移标志更新指令后,可以为上述待处理日志文件中的、对应上述待发送信息的最后一条信息设置更新偏移标志。上述更新偏移标志可以用于标记待处理日志文件中下一次待读取信息的起点位置。After the
进一步参考图5,作为对上述各图所示方法的实现,本公开提供了一种用于发送信息的装置的一个实施例,该装置实施例与图2所示的方法实施例相对应,该装置具体可以应用于各种电子设备中。Referring further to FIG. 5 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an apparatus for sending information. The apparatus embodiment corresponds to the method embodiment shown in FIG. 2 . The device can be specifically applied to various electronic devices.
如图5所示,本实施例的用于发送信息的装置500可以包括:第一信息发送单元501和第二信息发送单元502。其中,第一信息发送单元501,响应于缓冲队列中存在待发送包,将上述缓冲队列中的待发送包封装成信息包,被配置成将上述信息包发送至信息服务器,其中,上述待发送包由信息采集协程发送至缓冲队列,上述信息采集协程用于从待处理日志文件中采集信息,并将采集的信息封装成待发送包;第二信息发送单元502,响应于接收到上述信息服务器发来的对应上述信息包的信息存储成功消息,被配置成向上述信息采集协程发送偏移标志更新指令,上述偏移标志更新指令用于指示上述信息采集协程采集待处理日志文件中信息的起点。As shown in FIG. 5 , the
在本实施例的一些可选的实现方式中,用于发送信息的装置500还可以包括:信息采集协程设置单元(图中未示出),响应于检测到至少一个待处理日志文件,被配置成为上述至少一个待处理日志文件中的每个待处理日志文件设置信息采集协程和上述信息采集协程的信息采集周期。In some optional implementations of this embodiment, the
在本实施例的一些可选的实现方式中,上述待发送包的名称包括上述待处理日志文件的名称以及偏移标识的名称,上述偏移标识用于标记待处理日志文件中待读取信息的起点位置,以及In some optional implementations of this embodiment, the name of the packet to be sent includes the name of the log file to be processed and the name of an offset identifier, and the offset identifier is used to mark the information to be read in the log file to be processed the starting position of , and
上述第一信息发送单元501可以包括:序列划分子单元(图中未示出)和信息包封装子单元(图中未示出)。其中,序列划分子单元被配置成根据上述待发送包的名称对上述缓冲队列中的待发送包进行划分,得到至少一个待发送包序列,其中,待发送包序列内的待发送包属于同一待处理日志文件,待发送包序列内的待发送包按照偏移标识的名称排序;信息包封装子单元,对于上述至少一个待发送包序列中的待发送包序列,被配置成将该待发送包序列内的待发送包封装成信息包,并为信息包设置信息包名称。The above-mentioned first
在本实施例的一些可选的实现方式中,上述信息包封装子单元可以包括:信息包封装模块(图中未示出),被配置成按照顺序从上述待发送包序列中提取待发送包封装成信息包。In some optional implementations of this embodiment, the above-mentioned information packet encapsulation subunit may include: an information packet encapsulation module (not shown in the figure), configured to sequentially extract the to-be-sent packets from the above-mentioned to-be-sent packet sequence Encapsulated into packets.
在本实施例的一些可选的实现方式中,上述信息包封装子单元可以包括:信息包名称设置模块(图中未示出),被配置成根据信息包内待发送包的偏移标识的名称为信息包设置信息包名称。In some optional implementations of this embodiment, the above-mentioned information packet encapsulation subunit may include: an information packet name setting module (not shown in the figure), which is configured to Name Sets the package name for the package.
在本实施例的一些可选的实现方式中,上述第一信息发送单元501可以包括:信息包划分子单元(图中未示出)和第一信息发送子单元(图中未示出)。其中,信息包划分子单元,响应于信息包的数据量大于数据量阈值,被配置成将上述信息包划分为不超过上述数据量阈值的至少两个子信息包;第一信息发送子单元被配置成将上述至少两个子信息包中的子信息包发送至信息服务器。In some optional implementations of this embodiment, the above-mentioned first
进一步参考图6,作为对上述各图所示方法的实现,本公开提供了一种用于发送信息的装置的一个实施例,该装置实施例与图4所示的方法实施例相对应,该装置具体可以应用于各种电子设备中。Referring further to FIG. 6 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an apparatus for sending information. The apparatus embodiment corresponds to the method embodiment shown in FIG. 4 . The device can be specifically applied to various electronic devices.
如图6所示,本实施例的用于发送信息的装置600可以包括:偏移标识获取单元601、待发送信息获取单元602和第三信息发送单元603。其中,偏移标识获取单元601,被配置成获取待处理日志文件的偏移标识,上述偏移标识用于标记待处理日志文件中待读取信息的起点位置;待发送信息获取单元602,被配置成将上述偏移标志指示的待读取信息作为信息起点,从上述待处理日志文件中获取待发送信息;第三信息发送单元603,被配置成将上述待发送信息封装成待发送包发送至缓冲队列。As shown in FIG. 6 , the apparatus 600 for sending information in this embodiment may include: an offset identifier obtaining unit 601 , a to-be-sent information obtaining unit 602 , and a third information sending unit 603 . The offset identification obtaining unit 601 is configured to obtain the offset identification of the log file to be processed, and the offset identification is used to mark the starting point of the information to be read in the log file to be processed; the information obtaining unit 602 to be sent is is configured to use the information to be read indicated by the above offset flag as the information starting point, and obtain the information to be sent from the above log file to be processed; the third information sending unit 603 is configured to encapsulate the above information to be sent into packets to be sent for sending to the buffer queue.
在本实施例的一些可选的实现方式中,上述偏移标识获取单元601可以包括:日志查询子单元(图中未示出)和偏移标识获取子单元(图中未示出)。其中,日志查询子单元被配置成间隔设定的信息采集周期查询待处理日志文件;偏移标识获取子单元,响应于上述待处理日志文件存在的更新信息,被配置成获取上述待处理日志文件的偏移标识。In some optional implementations of this embodiment, the above offset identifier acquiring unit 601 may include: a log query subunit (not shown in the figure) and an offset identifier acquiring subunit (not shown in the figure). The log query subunit is configured to query the log files to be processed at the interval set information collection period; the offset identification acquisition subunit is configured to obtain the log files to be processed in response to the update information existing in the log files to be processed. offset identifier.
在本实施例的一些可选的实现方式中,上述偏移标识获取子单元可以包括:偏移标识获取模块(图中未示出),响应于上述更新信息的信息量大于设定信息量阈值,被配置成获取上述待处理日志文件的偏移标识。In some optional implementations of this embodiment, the above-mentioned offset identifier acquisition subunit may include: an offset identifier acquisition module (not shown in the figure), in response to the information amount of the above-mentioned update information being greater than the set information amount threshold , which is configured to obtain the offset identifier of the log file to be processed.
在本实施例的一些可选的实现方式中,上述待处理日志文件内的信息设置有对应的时间信息,以及,上述待发送信息获取单元602可以包括:待发送信息获取子单元(图中未示出),被配置成将上述偏移标志指示的待读取信息作为信息起点,将上述待处理日志文件中时间信息最后的信息的作为信息终点,将上述信息起点和信息终点之间的信息设置为待发送信息。In some optional implementations of this embodiment, the information in the log file to be processed is set with corresponding time information, and the information to be sent information acquisition unit 602 may include: a to-be-sent information acquisition subunit (not shown in the figure). shown), is configured to use the information to be read indicated by the above-mentioned offset flag as the information starting point, use the last information of the time information in the above-mentioned log file to be processed as the information ending point, and use the information between the information starting point and the information ending point. Set as pending message.
在本实施例的一些可选的实现方式中,上述第三信息发送单元603可以包括:第三信息发送子单元(图中未示出),被配置成将上述偏移标识和待发送信息封装成待发送包,并为上述待发送包设置名称,上述名称包括上述待处理日志文件的名称以及偏移标识的名称。In some optional implementations of this embodiment, the third information sending unit 603 may include: a third information sending subunit (not shown in the figure), configured to encapsulate the above offset identifier and the information to be sent A package to be sent is formed, and a name is set for the package to be sent, where the name includes the name of the log file to be processed and the name of the offset identifier.
在本实施例的一些可选的实现方式中,用于发送信息的装置600还可以包括:偏移标志更新单元(图中未示出),响应于接收到偏移标志更新指令,被配置成为上述待处理日志文件中的、对应上述待发送信息的最后一条信息设置更新偏移标志,上述更新偏移标志用于标记待处理日志文件中下一次待读取信息的起点位置。In some optional implementations of this embodiment, the apparatus 600 for sending information may further include: an offset flag update unit (not shown in the figure), configured to be, in response to receiving the offset flag update instruction, In the log file to be processed, an update offset flag is set corresponding to the last piece of information to be sent, and the update offset flag is used to mark the starting position of the next information to be read in the log file to be processed.
本实施例还提供了一种电子设备,包括:一个或多个处理器;存储器,其上存储有一个或多个程序,当上述一个或多个程序被上述一个或多个处理器执行时,使得上述一个或多个处理器执行上述的用于发送信息的方法。This embodiment also provides an electronic device, including: one or more processors; a memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, The above-mentioned one or more processors are caused to perform the above-mentioned method for transmitting information.
本实施例还提供了一种计算机可读介质,其上存储有计算机程序,该程序被处理器执行时实现上述的用于发送信息的方法。This embodiment also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, implements the above-mentioned method for sending information.
下面参考图7,其示出了适于用来实现本公开的实施例的电子设备(例如,图1中的日志采集设备105)的计算机系统700的结构示意图。图7示出的电子设备仅仅是一个示例,不应对本公开的实施例的功能和使用范围带来任何限制。Referring next to FIG. 7 , it shows a schematic structural diagram of a
如图7所示,电子设备700可以包括处理装置(例如中央处理器、图形处理器等)701,其可以根据存储在只读存储器(ROM)702中的程序或者从存储装置708加载到随机访问存储器(RAM)703中的程序而执行各种适当的动作和处理。在RAM 703中,还存储有电子设备700操作所需的各种程序和数据。处理装置701、ROM 702以及RAM 703通过总线704彼此相连。输入/输出(I/O)接口705也连接至总线704。As shown in FIG. 7 , an
通常,以下装置可以连接至I/O接口705:包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置706;包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置707;包括例如磁带、硬盘等的存储装置708;以及通信装置709。通信装置709可以允许电子设备700与其他设备进行无线或有线通信以交换数据。虽然图7示出了具有各种装置的电子设备700,但是应理解的是,并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。图7中示出的每个方框可以代表一个装置,也可以根据需要代表多个装置。Typically, the following devices can be connected to the I/O interface 705:
特别地,根据本公开的实施例,上文参考流程图描述的过程可以被实现为计算机软件程序。例如,本公开的实施例包括一种计算机程序产品,其包括承载在计算机可读介质上的计算机程序,该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中,该计算机程序可以通过通信装置709从网络上被下载和安装,或者从存储装置708被安装,或者从ROM 702被安装。在该计算机程序被处理装置701执行时,执行本公开的实施例的方法中限定的上述功能。In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the
需要说明的是,本公开的实施例上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开的实施例中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开的实施例中,计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输,包括但不限于:电线、光缆、RF(射频)等等,或者上述的任意合适的组合。It should be noted that, the above-mentioned computer-readable medium in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the foregoing two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Rather, in embodiments of the present disclosure, a computer-readable signal medium may include a data signal in baseband or propagated as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.
上述计算机可读介质可以是上述电子设备中所包含的;也可以是单独存在,而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序,当上述一个或者多个程序被该电子设备执行时,使得该电子设备:响应于缓冲队列中存在待发送包,将上述缓冲队列中的待发送包封装成信息包,将上述信息包发送至信息服务器,其中,上述待发送包由信息采集协程发送至缓冲队列,上述信息采集协程用于从待处理日志文件中采集信息,并将采集的信息封装成待发送包;响应于接收到上述信息服务器发来的对应上述信息包的信息存储成功消息,向上述信息采集协程发送偏移标志更新指令,上述偏移标志更新指令用于指示上述信息采集协程采集待处理日志文件中信息的起点。The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or may exist alone without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic equipment, the electronic equipment: in response to the existence of the to-be-sent packets in the buffer queue, the to-be-sent packets in the above-mentioned buffer queue are stored. The packet is encapsulated into an information packet, and the above-mentioned information packet is sent to the information server, wherein the above-mentioned packet to be sent is sent to the buffer queue by the information acquisition coroutine, and the above-mentioned information acquisition coroutine is used to collect information from the log file to be processed, and collect the information from the log file. The information is encapsulated into a packet to be sent; in response to receiving the information storage success message corresponding to the above-mentioned information packet sent by the above-mentioned information server, an offset flag update instruction is sent to the above-mentioned information collection coroutine, and the above-mentioned offset flag update instruction is used to indicate The above information collection coroutine collects the starting point of the information in the log file to be processed.
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的实施例的操作的计算机程序代码,上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for carrying out operations of embodiments of the present disclosure may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, and This includes conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).
附图中的流程图和框图,图示了按照本公开的各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分,该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意,在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个接连地表示的方框实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这依所涉及的功能而定。也要注意的是,框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或操作的专用的基于硬件的系统来实现,或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.
描述于本公开的实施例中所涉及到的单元可以通过软件的方式实现,也可以通过硬件的方式来实现。所描述的单元也可以设置在处理器中,例如,可以描述为:一种处理器包括第一信息发送单元和第二信息发送单元。其中,这些单元的名称在某种情况下并不构成对该单元本身的限定,例如,第一信息发送单元还可以被描述为“用于向信息服务器发送日志信息的单元”。The units involved in the embodiments of the present disclosure may be implemented in software or hardware. The described unit may also be provided in the processor, for example, it may be described as: a processor includes a first information sending unit and a second information sending unit. Wherein, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the first information sending unit may also be described as "a unit for sending log information to an information server".
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解,本公开中所涉及的发明范围,并不限于上述技术特征的特定组合而成的技术方案,同时也应涵盖在不脱离上述发明构思的情况下,由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, and should also cover, without departing from the above-mentioned inventive concept, the above-mentioned technical features or Other technical solutions formed by any combination of its equivalent features. For example, a technical solution is formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) with similar functions.
| Application Number | Priority Date | Filing Date | Title | 
|---|---|---|---|
| CN201910962981.0ACN110719233B (en) | 2019-10-11 | 2019-10-11 | Method and device for transmitting information | 
| Application Number | Priority Date | Filing Date | Title | 
|---|---|---|---|
| CN201910962981.0ACN110719233B (en) | 2019-10-11 | 2019-10-11 | Method and device for transmitting information | 
| Publication Number | Publication Date | 
|---|---|
| CN110719233Atrue CN110719233A (en) | 2020-01-21 | 
| CN110719233B CN110719233B (en) | 2023-10-31 | 
| Application Number | Title | Priority Date | Filing Date | 
|---|---|---|---|
| CN201910962981.0AActiveCN110719233B (en) | 2019-10-11 | 2019-10-11 | Method and device for transmitting information | 
| Country | Link | 
|---|---|
| CN (1) | CN110719233B (en) | 
| Publication number | Priority date | Publication date | Assignee | Title | 
|---|---|---|---|---|
| CN111240952A (en)* | 2020-02-28 | 2020-06-05 | 北京百度网讯科技有限公司 | Log collection method and device | 
| CN111614753A (en)* | 2020-05-20 | 2020-09-01 | 京东数字科技控股有限公司 | Method, system and device for sending log | 
| Publication number | Priority date | Publication date | Assignee | Title | 
|---|---|---|---|---|
| JP2000284995A (en)* | 1999-03-30 | 2000-10-13 | Fujitsu Ltd | Data processing device and recording medium | 
| CN106991177A (en)* | 2017-04-06 | 2017-07-28 | 广东亿迅科技有限公司 | The nondestructive collection method and its system of real-time logs under a kind of distributed environment | 
| CN108306771A (en)* | 2018-02-09 | 2018-07-20 | 腾讯科技(深圳)有限公司 | Log reporting method, apparatus and system | 
| CN108399186A (en)* | 2018-01-12 | 2018-08-14 | 联动优势科技有限公司 | A kind of collecting method and device | 
| CN109327351A (en)* | 2018-09-12 | 2019-02-12 | 拉扎斯网络科技(上海)有限公司 | Method and device for collecting log data in real time, electronic equipment and storage medium | 
| Publication number | Priority date | Publication date | Assignee | Title | 
|---|---|---|---|---|
| JP2000284995A (en)* | 1999-03-30 | 2000-10-13 | Fujitsu Ltd | Data processing device and recording medium | 
| CN106991177A (en)* | 2017-04-06 | 2017-07-28 | 广东亿迅科技有限公司 | The nondestructive collection method and its system of real-time logs under a kind of distributed environment | 
| CN108399186A (en)* | 2018-01-12 | 2018-08-14 | 联动优势科技有限公司 | A kind of collecting method and device | 
| CN108306771A (en)* | 2018-02-09 | 2018-07-20 | 腾讯科技(深圳)有限公司 | Log reporting method, apparatus and system | 
| CN109327351A (en)* | 2018-09-12 | 2019-02-12 | 拉扎斯网络科技(上海)有限公司 | Method and device for collecting log data in real time, electronic equipment and storage medium | 
| Publication number | Priority date | Publication date | Assignee | Title | 
|---|---|---|---|---|
| CN111240952A (en)* | 2020-02-28 | 2020-06-05 | 北京百度网讯科技有限公司 | Log collection method and device | 
| CN111240952B (en)* | 2020-02-28 | 2023-07-21 | 北京百度网讯科技有限公司 | Log collection method and device | 
| CN111614753A (en)* | 2020-05-20 | 2020-09-01 | 京东数字科技控股有限公司 | Method, system and device for sending log | 
| CN111614753B (en)* | 2020-05-20 | 2023-04-18 | 京东科技控股股份有限公司 | Method, system and device for sending log | 
| Publication number | Publication date | 
|---|---|
| CN110719233B (en) | 2023-10-31 | 
| Publication | Publication Date | Title | 
|---|---|---|
| US20220394316A1 (en) | Message sending method and device, readable medium and electronic device | |
| CN111784380B (en) | Advertisement putting attribution method and device | |
| CN112199174B (en) | Message sending control method and device, electronic equipment and computer readable storage medium | |
| CN103401934A (en) | Method and system for acquiring log data | |
| CN111273999B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN109743209B (en) | Method and apparatus for transmitting information | |
| CN112015816A (en) | Data synchronization method, device, medium and electronic equipment | |
| WO2021073054A1 (en) | Data processing method, apparatus and device, and storage medium | |
| CN110719233B (en) | Method and device for transmitting information | |
| CN111240952B (en) | Log collection method and device | |
| CN109819026B (en) | Method and device for transmitting information | |
| CN115225586B (en) | Data packet transmitting method, device, equipment and computer readable storage medium | |
| CN111767185B (en) | Data embedding method and device | |
| US20250227166A1 (en) | Data uploading method and apparatus, device, and medium | |
| CN116980640A (en) | Live video processing method, device, equipment and storage medium | |
| CN112311842A (en) | Method and apparatus for information interaction | |
| CN107342981B (en) | Sensor data transmission method and device and virtual reality head-mounted equipment | |
| US20230418470A1 (en) | Data processing method and apparatus, and electronic device | |
| CN114035861B (en) | Cluster configuration method, device, electronic device and computer readable medium | |
| CN110245120A (en) | Stream computing system and log data processing method of stream computing system | |
| CN112311833B (en) | Data updating method and device | |
| CN116560918A (en) | A pressure testing method, device, equipment and medium | |
| CN115022328A (en) | Server cluster, server cluster testing method and device and electronic equipment | |
| CN115431901A (en) | A signal acquisition method, device, electronic equipment and storage medium | |
| CN109614137B (en) | Software version control method, device, equipment and medium | 
| Date | Code | Title | Description | 
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |