HK1225485B - A method and a device for decoding data streams in reconfigurable platforms - Google Patents

Description

Translated fromChinese

用于在可重构平台中解码数据流的方法和设备Method and apparatus for decoding data stream in a reconfigurable platform

技术领域Technical Field

本发明大致上涉及一种用于处理市场数据的数据处理系统，更具体而言，本发明涉及一种用于在可重构平台中解码数据流的方法和设备。The present invention generally relates to a data processing system for processing market data, and more particularly, to a method and apparatus for decoding a data stream in a reconfigurable platform.

背景技术Background Art

随着在金融业中的对于更快地处理大量数据的需求的增长，基于依赖通用CPU的集群的数据处理系统显示出了很多限制。实际上，即使集群方法涉及廉价的硬件并且提供简化的研发工具，其也具有很多随着对于高性能计算的需求增加而更加显著的限制：高电力消耗、昂贵的维护成本、数据中心所需的大量空间。此外，集群所获得的整体性能并不随集群数量成比例地增加。不同于集群方法，使用有限数量的配备有FPGA的机器的、基于FPGA的数据处理系统能够以大吞吐量来并行执行复杂的任务。因此，该硬件方案对于金融和投资业领域中的应用的研发尤其适用，在这些领域中，快速计算是保持竞争力的关键。As the demand for faster processing of large amounts of data grows in the financial industry, data processing systems based on clusters that rely on general-purpose CPUs have shown many limitations. Indeed, even though cluster approaches involve inexpensive hardware and offer simplified R&D tools, they also have many limitations that become more pronounced as the demand for high-performance computing increases: high power consumption, expensive maintenance costs, and the large amount of space required in data centers. Furthermore, the overall performance achieved by a cluster does not scale proportionally with the number of clusters. Unlike cluster approaches, FPGA-based data processing systems, which use a limited number of FPGA-equipped machines, can execute complex tasks in parallel with high throughput. Therefore, this hardware solution is particularly suitable for the development of applications in the financial and investment industries, where fast computing is key to staying competitive.

FPGA(acronym for Field-programmable gate array，现场可编程门阵列的缩写)指的是可以在制造之后进行配置的集成电路。该配置一般指定为使用“硬件描述语言”(Hardware description language，HDL)。FPGA包含大量的可编程逻辑部件(“逻辑块”)，以及使得这些块能够“连线在一起”的可重构的互联的体系。逻辑块可以被配置为执行复杂组合逻辑，或仅执行简单的基本逻辑运算(布尔AND、OR、NAND、XOR等)。因为FPGA可以执行并行计算，所以仅仅在几个时钟周期内，对于多个独立的输入，可以同时执行相同的算法。因此，FPGA尤其适于非常块地执行复杂计算。An FPGA (acronym for Field-programmable gate array) refers to an integrated circuit that can be configured after manufacture. This configuration is typically specified using a "hardware description language" (HDL). An FPGA contains a large number of programmable logic components ("logic blocks"), as well as a reconfigurable interconnect system that enables these blocks to be "wired together." The logic blocks can be configured to perform complex combinational logic, or just simple basic logic operations (Boolean AND, OR, NAND, XOR, etc.). Because FPGAs can perform parallel calculations, the same algorithm can be executed simultaneously for multiple independent inputs in just a few clock cycles. FPGAs are therefore particularly well-suited for performing complex calculations very quickly.

基于这些原因，越来越多的市场数据处理系统使用FPGA来设计。For these reasons, more and more market data processing systems are designed using FPGAs.

现有的市场数据处理系统从外部源(例如交易所)接收数据，向其订阅者(例如，在工作站的交易商)发布感兴趣的金融数据，并且将交易数据发送到各个交易所或其他场所。Existing market data processing systems receive data from external sources (eg, exchanges), publish financial data of interest to their subscribers (eg, traders at their workstations), and send trade data to various exchanges or other venues.

这些市场数据处理系统一般包括至少一个解码器，其与用于处理以给定格式(FAST、FIX、二进制)的实时数据流的供给源进行交互，并且对其进行解码，将数据流从特定源的格式转换为内部格式(数据标准化过程)。根据每个数据供给中的消息结构，解码器以特定的操作处理每个域值(field value)，将缺失数据填充以其在缓存中记录的值和状态，并且将其映射到系统所使用的格式。These market data processing systems typically include at least one decoder that interacts with a feed of real-time data streams in a given format (FAST, FIX, binary) and decodes the data stream, converting it from the source-specific format to the internal format (a data normalization process). Based on the message structure in each data feed, the decoder processes each field value with a specific operation, filling in missing data with the value and status recorded in the buffer, and mapping it to the format used by the system.

当前，输入数据流的解码由软件或硬件执行，其执行方式是纯粹顺序的，而不进行任何并行化。现有的以软件执行解码的解码器经常受制于带宽限制，这是因为解码器的处理器不能足够快地解码数据包。这是由于，软件解码器需要解码每个消息以确定其是否涉及有关应用的证券(instrument)。此外，当以硬件进行剩下的处理时，需要两种从硬件到软件以及从软件到硬件的转移。相比于典型的处理时间，这些转移非常耗时，而且增加了大量的延迟。Currently, decoding of input data streams is performed in software or hardware in a purely sequential manner without any parallelization. Existing decoders that perform decoding in software often suffer from bandwidth limitations because the decoder's processor cannot decode packets quickly enough. This is because software decoders need to decode each message to determine whether it refers to an instrument for the application. Furthermore, when the remaining processing is performed in hardware, two transfers are required, from hardware to software and back again. These transfers are very time-consuming and add significant latency compared to typical processing times.

在最近过去的几年，市场数据速率大大增长，峰值接近一百万个消息每秒。随着市场数据速率持续增长，高速、极低延迟且可靠的市场数据处理系统对于金融机构的成功变得愈发重要。具体而言，当前存在着对于提供下述高性能解码器的需求：其使用不依赖于所处理的市场的标准化指令，能够处理上至10Gb/s的供给至订单管理核心的市场数据，同时仍具有最低的可能延迟。Market data rates have increased significantly over the past few years, reaching peaks approaching one million messages per second. As market data rates continue to grow, high-speed, extremely low-latency, and reliable market data processing systems are becoming increasingly critical to the success of financial institutions. Specifically, there is a need for a high-performance decoder capable of processing up to 10 Gb/s of market data fed to order management cores, using standardized instructions independent of the market being processed, while still maintaining the lowest possible latency.

此外，市场数据格式，尤其是FAST中的市场数据格式，演化得十分频繁。这对于传统的软件解码器并不产生任何重大的问题，通常可以容易地修改软件解码器。对于FAST格式，交易所提供了更新的模板文件，并且软件动态地加载该文件，或者软件的代码(或代码的一部分)自动地从这些模板再生成。Furthermore, market data formats, especially those in FAST, evolve frequently. This does not pose any significant problems for traditional software decoders, which can usually be easily modified. For the FAST format, exchanges provide updated template files, and the software dynamically loads these files, or the software code (or portions of it) is automatically regenerated from these templates.

然而，对于使用可重构平台(FPGA)的解码器，有效率地适应这样的格式改变是困难的。实际上，虽然通用CPU可以容易地更新以执行任何任务，但是为了特定的任务而一旦编程了FPGA，更新FPGA使其可以执行另一个任务是非常复杂的。这需要再次对FPGA进行编程，而这是昂贵且复杂的。However, for decoders using reconfigurable platforms (FPGAs), efficiently adapting to such format changes is difficult. While a general-purpose CPU can be easily updated to perform any task, once an FPGA has been programmed for a specific task, updating it to perform another task is very complex. This requires reprogramming the FPGA, which is expensive and complex.

发明内容Summary of the Invention

为了解决这些问题和其他问题，提供如所附独立权利要求1所限定的用于解码输入市场数据流的设备，以及如所附权利要求15所限定的用于解码输入市场数据流的方法。从属权利要求中限定了优选的实施方案。In order to solve these and other problems there is provided an apparatus for decoding an input market data stream as defined in the appended independent claim 1, and a method for decoding an input market data stream as defined in the appended claim 15. Preferred embodiments are defined in the dependent claims.

本发明因此提供了高性能解码器，其能够处理上至10GB/s的市场数据供给，以便以不依赖所处理的市场的标准化指令供给至订单管理核心，同时仍具有可能的最低的延迟，以及使用和更新软件解码器的便利性。The present invention therefore provides a high performance decoder capable of processing up to 10 GB/s of market data feed for feeding to the order management core in standardized instructions that are independent of the market being processed, while still having the lowest possible latency and convenience of using and updating the software decoder.

根据本发明的实施方案的解码器可以显然地适应于数据格式的演化。这样的适应可以通过更新描述文件、重新编译描述文件并且提供固件的新版本以便下载到可重新配置平台而容易地执行，该描述文件是以诸如XML(可扩展标记语言)的格式写成的。Decoders according to embodiments of the present invention can readily adapt to evolving data formats. Such adaptation can be easily performed by updating a description file written in a format such as XML (Extensible Markup Language), recompiling the description file, and providing a new version of the firmware for download to the reconfigurable platform.

通过验证说明书附图和具体实施方式，本发明的其他益处对于本领域技术人员将变得明显。应当注意，任何额外的益处也包含于本文。Other benefits of the present invention will become apparent to those skilled in the art by examining the accompanying drawings and detailed description. It should be noted that any additional benefits are also encompassed herein.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

参考所附附图，现在将通过示例的方式来描述本发明的实施方案，在附图中，相似的附图标记表示相似的元素，而且在附图中：Embodiments of the present invention will now be described by way of example with reference to the accompanying drawings, in which like reference numerals represent like elements, and in which:

-图1展示出了包括有根据本发明的实施方案的解码设备的示例性数据处理架构；FIG1 shows an exemplary data processing architecture including a decoding device according to an embodiment of the present invention;

-图2显示了根据本发明的特定实施方案的解码器架构；- Figure 2 shows a decoder architecture according to a particular embodiment of the invention;

-图3是示出了根据本发明的实施方案的生成解码器引擎的框图；- FIG3 is a block diagram illustrating a generative decoder engine according to an embodiment of the present invention;

-图4显示了根据本发明的特定实施方案的令牌解析器(tokenizer)的架构；- Figure 4 shows the architecture of a tokenizer according to a particular embodiment of the present invention;

-图5是根据本发明的特定实施方案的对输入流进行解码所执行的步骤的流程图；- FIG5 is a flow chart of the steps performed to decode an input stream according to a particular embodiment of the invention;

-图6显示了根据本发明的特定实施方案的示例性的有限状态机；- FIG. 6 shows an exemplary finite state machine according to a particular embodiment of the present invention;

以及as well as

-图7是示出了根据本发明的特定实施方案的示例性的FPGA实施方式的框图。- Figure 7 is a block diagram illustrating an exemplary FPGA implementation according to a specific embodiment of the present invention.

具体实施方式DETAILED DESCRIPTION

参考图1，所显示的是，设置为获取并处理市场数据的示例性数据处理系统100。1 , shown is an exemplary data processing system 100 configured to acquire and process market data.

如同本文使用的，术语“市场数据”指从多个外部源接收的数据流形式的数据，包括报价以及与资产净值、固定收入、金融衍生产品、货币等相关联的与交易相关的数据。As used herein, the term "market data" refers to data received in the form of data streams from a plurality of external sources, including quotes and transaction-related data associated with equity, fixed income, financial derivatives, currencies, and the like.

数据处理系统100包括至少一个市场数据包解码器10(也称为“解码器”或“解码设备”)，其与供给源交互以便处理市场数据供给从而将其解码，该市场数据供给是根据来自交易网络1的任何特定源的协议而接收的。The data processing system 100 includes at least one market data packet decoder 10 (also referred to as a "decoder" or "decoding device") that interacts with a feed source to process and decode a market data feed received according to a protocol from any particular source of the trading network 1.

更具体而言，解码设备10被配置为，接收诸如UDP有效载荷(用户数据报协议的缩写)或TCP有效载荷(传输控制协议的缩写)的输入消息，将其解码为消息，并且基于所解码的消息来输出标准化的指令8。More specifically, the decoding device 10 is configured to receive an input message such as a UDP payload (acronym for User Datagram Protocol) or a TCP payload (acronym for Transmission Control Protocol), decode it into a message, and output standardized instructions 8 based on the decoded message.

由解码设备10提供的输出指令8可以供给到订单管理系统12。订单管理系统12包括至少一个存储器，其用于存储关于每个订单的细节，以便在需要时对其进行检索。The output instructions 8 provided by the decoding device 10 may be fed to an order management system 12. The order management system 12 comprises at least one memory for storing details about each order so that they can be retrieved when required.

系统100可以进一步包括额度聚合类(Limits Aggregation)和账簿建立单元(Book Building unit)13，其用于将等待进入订单账簿的订单聚类，对于每个展示证券订单的列表，该列表可能是聚类为额度的并且可能是按价格排序的。一般而言，客户端应用15实质上需要访问账簿的第一额度。可替选地，客户端应用可以直接地访问订单。System 100 may further include a limits aggregation class and a book building unit 13, which is used to cluster orders waiting to be entered into the order book. For each list showing securities orders, the list may be clustered by limits and sorted by price. Generally speaking, client application 15 essentially needs to access the first limit in the book. Alternatively, the client application can directly access the order.

如同本文所使用的，术语“订单”或“额度订单”或“市场订单”指的是，以特定的限定价格或更好的价格、或以市场订单的市场价格来买入或卖出给定数目的金融证券的订单。As used herein, the term "order" or "line order" or "market order" refers to an order to buy or sell a given amount of a financial security at a specific limit price or better, or at the market price for a market order.

此外，订单账簿指的是金融证券(例如，股票)的未付款额度订单的电子集合。如同本文所使用的，术语“额度”指的是对应于一个或几个订单的订单账簿中的“排列”或“入口”。当其对应于几个订单时，其还可以称为“聚合额度”。额度是根据价格聚类的，即，具有相同价格的全部订单的数目相加形成了额度的数目。聚类额度还可以具有“订单计数”属性，该属性反映聚类在该额度中的订单的数量。额度在账簿中的位置被称为“排数”或“水平”。In addition, the order book refers to an electronic collection of outstanding quota orders for financial securities (e.g., stocks). As used herein, the term "quota" refers to an "arrangement" or "entry" in the order book corresponding to one or several orders. When it corresponds to several orders, it may also be referred to as an "aggregate quota." Quotas are clustered according to price, i.e., the number of all orders with the same price is added together to form the number of quotas. Clustered quotas may also have an "order count" attribute that reflects the number of orders clustered in that quota. The position of a quota in the book is referred to as a "row" or "level."

如图所示，数据处理系统100还可以包括消息分发和传输单元14，其用于将经处理的数据格式化为消息，并且将其分发到所选择的客户端应用15以供进一步的处理和决策。客户端应用15可以位于不同的服务器，使得消息传输可以在网络上完成。As shown in the figure, the data processing system 100 may further include a message distribution and transmission unit 14, which is used to format the processed data into messages and distribute them to the selected client application 15 for further processing and decision-making. The client application 15 can be located on different servers so that message transmission can be completed over the network.

在许多数据处理域中，需要提高数据处理速度。然而，现代数据处理系统面对的数据量正在增长。数据处理链的第一步骤包括数据获取阶段以及数据包解码阶段，数据获取阶段在网络应用中包括网络(一般为UDP或TCP)获取2，并且一般由NIC(网络接口卡，网卡)和操作系统的网络堆栈执行。In many data processing domains, there is a need to increase data processing speed. However, the amount of data handled by modern data processing systems is constantly increasing. The first step in the data processing chain includes the data acquisition phase and the packet decoding phase. In network applications, the data acquisition phase involves network (typically UDP or TCP) acquisition and is typically performed by the NIC (Network Interface Card) and the operating system's network stack.

由解码设备10执行的数据包解码阶段取决于所引入数据的格式，而引入的数据自身又取决于特定的应用。例如，在用于处理由金融机构交换的市场数据的数据处理系统中，数据包解码阶段取决于市场，因为每个市场有其自己的数据格式。The packet decoding stage performed by the decoding device 10 depends on the format of the incoming data, which itself depends on the specific application. For example, in a data processing system for processing market data exchanged by financial institutions, the packet decoding stage depends on the market, as each market has its own data format.

一般而言，市场数据供给使用三种格式：Generally speaking, market data feeds come in three formats:

-二进制格式，- binary format,

-基于FIX的格式，以及- a FIX-based format, and

-基于FAST的格式。-FAST-based format.

在二进制格式中(其一般用于资产净值市场)，所有的域是以二进制发送的，而且消息结构是固定的。这使得解码设备10能够使用这些结构来解析消息。消息一般封装在包中，其二进制的头部指示包中所包括的消息的数量。消息一般开始于指示其大小的域，使得解码设备可以基于大小信息而跳过其不进行处理的消息，并且可能进行某种完整性的检查。In a binary format (typically used in the net asset value market), all fields are sent in binary format, and the message structure is fixed. This allows the decoding device 10 to use this structure to parse the message. Messages are typically encapsulated in packets, with a binary header indicating the number of messages included in the packet. Messages typically begin with a field indicating their size, allowing the decoding device to skip messages it does not process based on the size information and possibly perform some kind of integrity check.

在一些市场应用中，在某些域中可以放置ASCII字符串(或较为少见的UNICODE字符串)。然而，当这些字符串具有固定长度时，由于在必要时对字符串的末端进行填充，可以按常规二进制域对其进行处理。In some market applications, ASCII strings (or less commonly, UNICODE strings) can be placed in certain fields. However, when these strings have a fixed length, they can be treated as regular binary fields because the end of the string is padded when necessary.

FIX表示金融信息交换(Financial Information eXchange)。在这种类型的格式中，各个域按ASCII编码，每个域前面是其域ID，并且每个域由SOH(Start Of Header，头部开始)字符分开。下面的示例表示FIX编码的消息，其中符号“|”表示SOH字符：FIX stands for Financial Information eXchange. In this format, fields are encoded in ASCII, each field is preceded by its field ID, and each field is separated by a SOH (Start of Header) character. The following example shows a FIX-encoded message, where the symbol "|" represents the SOH character:

“8＝FIX.4.2|9＝178|35＝8|49＝PHLX|56＝PERS|52＝20071123-05:30:00.000|"8＝FIX.4.2|9＝178|35＝8|49＝PHLX|56＝PERS|52＝20071123-05:30:00.000|

11＝ATOMNOCCC9990900|20＝3|150＝E|39＝E|55＝MSFT|167＝CS|54＝1|38＝15|11＝ATOMNOCCC9990900|20＝3|150＝E|39＝E|55＝MSFT|167＝CS|54＝1|38＝15|

40＝2|44＝15|58＝PHLX EQUITY TESTING|59＝0|47＝C|32＝0|31＝0|151＝15|40＝2|44＝15|58＝PHLX EQUITY TESTING|59＝0|47＝C|32＝0|31＝0|151＝15|

14＝0|6＝0|10＝128|”14=0|6=0|10=128|”

例如，上述消息的第一部分“8＝FIX.4.2”限定了域8的值，其是所使用的FIX协议的版本，而部分“35＝8”限定了域35的值，其表示消息类型。For example, the first part "8=FIX.4.2" of the above message defines the value of field 8, which is the version of the FIX protocol used, and the part "35=8" defines the value of field 35, which indicates the message type.

在FIX中，域不具有固定的大小，而且可以以几乎任何顺序呈现，除了某些具有强制顺序的域，例如消息头中的域，或重复的数据组中的域。In FIX, fields do not have a fixed size and can appear in almost any order, except for certain fields that have a mandatory order, such as fields in a message header, or fields in a repeated data group.

基于FIX的格式对于多数交易都是可用的，通常作为补充其专有格式的遗留格式。FIX-based formats are available for most transactions, often as a legacy format to supplement their proprietary formats.

FAST表示适应于STreaming的FIX(FIX Adapted for STreaming)。这是FIX协议的压缩变种，其被设计为，通过移除冗余信息来使用更少的带宽。域是按二进制编码的，但是具有“停止比特”：每个字节包含7个载荷比特以及一个称为“停止比特”的比特，该停止比特标记了域的结束。类似FIX，FAST的域从而具有可变的大小。FAST stands for FIX Adapted for Streaming. This is a compressed variant of the FIX protocol, designed to use less bandwidth by removing redundant information. Fields are encoded in binary, but with a "stop bit": each byte contains 7 payload bits and a single "stop bit" that marks the end of the field. Like FIX, FAST fields are therefore of variable size.

此外，在FAST中，域一直具有相同的顺序，但是对其应用了操作符，而且在某些情况下，可以从输入流完全移除域。Furthermore, in FAST, the fields are always in the same order, but operators are applied to them, and in some cases fields can be removed from the input stream entirely.

在期货和衍生品市场应用中，供给按FAST编码，交易可以提供一个或多个模板文件。这些模板描述消息的结构以及应用到每个域的操作符。例如，可以从流中完全移除具有保持不变的“增量”操作符的域。如果该相同的域增长了1，流中传送的值是1，该值被编码在使用停止比特的1个字节中，而不会发送域的整个值(这可能需要几个字节)。某些域表示“存在映射(presence map)”，并且可以用于编码其他域的存在。从而，当该域中的比特被设定为0时，其指示对应的可选的域不存在。一般而言，不会传送冗余信息，而且解码设备仍可以复原全部信息。In futures and derivatives market applications, where the feed is encoded in FAST, the exchange can provide one or more template files. These templates describe the structure of the message and the operators to be applied to each field. For example, a field with an "increment" operator that remains unchanged can be completely removed from the stream. If the same field is incremented by 1, the value transmitted in the stream is 1, which is encoded in 1 byte using a stop bit, rather than sending the entire value of the field (which may require several bytes). Some fields represent "presence maps" and can be used to encode the presence of other fields. Thus, when a bit in the field is set to 0, it indicates that the corresponding optional field does not exist. In general, no redundant information is transmitted and the decoding device can still recover the entire information.

FAST协议主要用于期货和衍生品交易。The FAST protocol is mainly used for futures and derivatives trading.

常规的FIX、FAST甚至二进制解码器一般根据串行处理来对输入流进行解码。每一个域一般是顺序地读取。在需要时可以应用操作符，在每个域之后可以采取关于解码之后的步骤的决策。Conventional FIX, FAST, and even binary decoders typically decode the input stream using a serial process. Each field is typically read sequentially. Operators can be applied as needed, and decisions can be made after each field regarding subsequent decoding steps.

在FIX协议中，消息中的域可以具有任何顺序，消息的结束是通过下一个消息的开始来进行检测的。类似地，对于重复的数据组，一个组的结束是通过另一个组的开始来进行检测的。因此，不能提前知道将要读取的域的数量，而且域必须一个接一个地读取。In the FIX protocol, fields within a message can be in any order, and the end of a message is detected by the start of the next message. Similarly, for repeated data groups, the end of one group is detected by the start of another. Therefore, the number of fields to be read cannot be known in advance, and fields must be read one by one.

在FAST协议中，每个字节被附加到当前域缓冲器，直到找到停止比特。然后向该域应用适当的操作符。然而，如果域不存在(即，对应的存在映射比特被设定为0)，没有数据可以读取而且该域必须被跳过。对于某些操作符也是如此，例如：In the FAST protocol, each byte is appended to the current field buffer until the stop bit is found. The appropriate operator is then applied to the field. However, if the field does not exist (i.e., the corresponding presence mapping bit is set to 0), no data can be read and the field must be skipped. This is also true for some operators, such as:

-“增量(delta)”操作符：当旧的和新的值相同时，增量操作符可以将存在映射比特设定为0，并且不在流中装入任何东西，- "delta" operator: when the old and new values are the same, the delta operator may set the presence map bit to 0 and not load anything into the stream,

-数据类型：“可选的十进制小数(decimals)”仅可以使用1个域；则指数在流中而且是空的，而且尾数不存在；将要读取的域的数量不能提前知道。- Data type: "optional decimals" may only use 1 field; the exponent is in the stream and is empty, and the mantissa is not present; the number of fields to be read cannot be known in advance.

对于二进制协议，每个消息的大小是提前知道的。然而，仍有一些构造是顺序的，例如对适当的消息类型的选择。For binary protocols, the size of each message is known in advance. However, some construction is still sequential, such as the selection of the appropriate message type.

当前，基于FIX或FAST协议或者使用二进制的消息的解码由软件或硬件执行，其执行方式纯粹是顺序的，而不进行任何并行化。Currently, decoding of messages based on the FIX or FAST protocols or using binary is performed by software or hardware in a purely sequential manner without any parallelization.

多数现有的解码设备是针对为特定市场而优化手写的，因此对于每个市场是不同的。某些现有的软件解码设备是从描述文件编译的，但是，主要的已知的用于FAST协议的软件解码器是使用模板(模板可能是预编译的)的通用解码器。然而，即使是在模板被预编译为某种形式的二进制代码的情况下，这些方法也不会达到相同的数据速率。Most existing decoding devices are handwritten and optimized for specific markets, and therefore are different for each market. Some existing software decoding devices are compiled from description files, but the main known software decoders for the FAST protocol are general-purpose decoders that use templates (which may be pre-compiled). However, even when the templates are pre-compiled into some form of binary code, these methods do not achieve the same data rates.

根据本发明的各个实施方案的解码设备10依赖提供特定地生成且优化的代码的代码生成方法，所述特定地生成且优化的代码使得能够达到比常规的通用解码器所获得的数据速率更高的数据速率。这样的根据本发明的实施方案的代码生成在硬件解码器的领域是不同寻常的。The decoding device 10 according to various embodiments of the present invention relies on a code generation method that provides a specifically generated and optimized code that enables data rates higher than those achieved by conventional general-purpose decoders. Such code generation according to embodiments of the present invention is unusual in the field of hardware decoders.

现在参考图2，其显示了根据本发明的特定实施方案的解码设备10(也称为解码器)的内部结构。Reference is now made to FIG. 2 , which shows the internal structure of a decoding device 10 (also referred to as a decoder) according to a particular embodiment of the present invention.

解码设备10被配置为，接收输入流2并且在其输出总线上生成输出指令8。输入流2是按给定的数据格式(其可以是基于FIX的、基于FAST的、二进制的)接收的，并且由市场(例如，提供二进制数据流的NASDAQ市场)提供。具体而言，输入流2可以以任何扩展了FAST模板规范的数据表示格式提供。The decoding device 10 is configured to receive an input stream 2 and generate output instructions 8 on its output bus. The input stream 2 is received in a given data format (which may be FIX-based, FAST-based, or binary) and is provided by a market (e.g., a NASDAQ market providing binary data streams). Specifically, the input stream 2 may be provided in any data representation format that extends the FAST template specification.

解码设备10包括引擎4，该引擎由编译器基于至少一个描述文件5生成。描述文件5描述了将由引擎4生成的指令。引擎4提供解码设备的代码中的大部分，所述解码设备适应于处理输入流2并且提供标准化的输出指令8。The decoding device 10 comprises an engine 4 generated by a compiler based on at least one description file 5. The description file 5 describes the instructions to be generated by the engine 4. The engine 4 provides the bulk of the code of the decoding device adapted to process an input stream 2 and provide standardized output instructions 8.

从而，解码器被进一步配置为，将来自市场的各种消息转换为输出指令8。对于订单驱动市场(例如，NASDAQ、BATS)，消息可以包括例如：Thus, the decoder is further configured to convert various messages from the market into output instructions 8. For order-driven markets (e.g., NASDAQ, BATS), the messages may include, for example:

-增加消息，其用于请求向订单账簿增加订单；-Add message, which is used to request to add an order to the order book;

-取消或删除消息，其用于请求从订单账簿删除(完全删除或部分删除)订单，- Cancel or Delete messages, which are used to request the removal (complete or partial) of an order from the order book,

-执行消息，其请求执行(完全执行或部分执行)来自订单账簿的订单，- an execution message, which requests the execution (full or partial) of an order from the order book,

-替换或修改消息，其用于请求修改订单账簿中包括的额度订单的一个或多个属性(例如，修改数目和/或价格)。- A replace or modify message, which is used to request modification of one or more attributes of a quota order included in the order book (e.g., modification of quantity and/or price).

对于“额度驱动”市场(例如，CME、EUREX)，消息可以进一步包括：For “quota-driven” markets (e.g., CME, EUREX), the message may further include:

-额度创建消息，其用以创建(根据其水平而索引的)额度，并且将该额度之下的全部额度向下移动，- a line creation message, which creates a line (indexed by its level) and moves all lines below it downwards,

-额度删除消息，其用以删除账簿中的(根据其水平而索引的)额度，并且将该额度之下的全部额度向上移动，以及- a line delete message, which deletes a line in the ledger (indexed by its level) and moves all lines below it upwards, and

-额度修改消息，其用以修改(根据其水平而索引的)额度。- Quota Modification message, which is used to modify a quota (indexed by its level).

对于“价格驱动”市场(例如，LIFFE、TSE)，消息还可以包括：价格更新消息，其用于更新根据其价格而索引的额度；如果还不存在已有该额度，这样的消息创建额度；如果其数目达到0，则将其删除。For "price driven" markets (e.g. LIFFE, TSE), messages may also include: price update messages, which are used to update quotas indexed by their prices; such messages create quotas if they do not already exist; and delete them if their number reaches 0.

这些消息由解码设备10变换为指令。输出指令8由总线承载，其可以包括多个信号，例如：These messages are transformed into instructions by the decoding device 10. The output instructions 8 are carried by a bus which may include a number of signals, for example:

-操作代码或操作码，其用于识别指令类型(例如，对于《订单驱动》流的增加、删除、替换等，或者对于“额度驱动”流的额度创建、额度删除等)；- Operation code or opcode, which is used to identify the instruction type (e.g., add, delete, replace, etc. for the "order driven" flow, or quota create, quota delete, etc. for the "quota driven" flow);

-证券标识符，其用于识别指令所涉及的证券；- a security identifier, which identifies the security to which the instruction relates;

-订单标识符，其用于识别指令所涉及的订单；- an order identifier, which is used to identify the order to which the instruction relates;

-价格和数目信息，其表示指令的价格和数目的参数；- price and quantity information, which indicates the price and quantity parameters of the instruction;

-其他依据指令的数据信号。-Other data signals based on instructions.

上述信号中的一些可以不使用。例如，订单标识符可以仅用于涉及订单的指令，证券标识符可以仅用于涉及特殊证券的指令，而且当证券是已知的时：用于删除特定订单的删除指令例如涉及在给定证券上的订单。市场一般不传送该证券，从而证券不会出现在输出指令总线8上。订单管理系统12则负责使用订单ID来找到证券ID。Some of the above signals may not be used. For example, the order identifier may be used only for instructions involving orders, the security identifier may be used only for instructions involving a specific security, and when the security is known, a delete instruction for deleting a specific order may refer to an order for a given security. The market generally does not transmit that security, so it does not appear on outgoing instruction bus 8. Order management system 12 is then responsible for using the order ID to find the security ID.

解码设备10可以进一步包括一组转换单元6。转换单元6被配置为，进一步标准化由引擎4输出的、已经尽可能地接近所需的标准化输出指令8的指令。实际上，描述文件5可能难以有效地描述某些操作，或者某些操作需要不能自动生成的特定地优化的代码。转换单元6可以包括例如下列单元：The decoding device 10 may further include a set of conversion units 6. The conversion units 6 are configured to further standardize the instructions output by the engine 4, which are already as close as possible to the required standardized output instructions 8. In practice, the description file 5 may be difficult to effectively describe certain operations, or some operations require specially optimized code that cannot be automatically generated. The conversion unit 6 may include, for example, the following units:

-ASCII至二进制转换器60：将二进制流中的某些域按ASCII编码，如同FIX流中的所有域一样。例如，当这样的域对应于数目时，将其转换为二进制格式可能是有利的，因为这使得能够向其应用算数操作。例如，当在其上发送输出指令8的总线需要某些域必须以整数发送时(例如，数目就是这样的情况)，可以应用上述转换。ASCII to binary converter 60: Certain fields in the binary stream are encoded in ASCII, as are all fields in the FIX stream. For example, when such fields correspond to numbers, it may be advantageous to convert them to binary format, as this enables arithmetic operations to be applied to them. This conversion may be applied, for example, when the bus on which the output instruction 8 is sent requires that certain fields must be sent as integers (for example, this is the case with numbers).

-价格格式转换器61：依据市场，市场数据流可以包含各种格式的价格，例如浮点、具有各种系数(x100、x10000…)的定点、与“刻点(ticks)大小”相关联的“刻点”(在该情况下，实际价格是通过将刻点的数量乘以刻点大小而获得的)。为了能够处理多个市场并且容易地比较其价格，可以将价格标准化为具有系数10^8的定点值。这使得能够在不损失任何信息的情况下对来自全部已知的市场的价格进行编码。Price Format Converter 61: Depending on the market, the market data stream may contain prices in various formats, such as floating point, fixed point with various coefficients (x100, x10000, ...), "ticks" associated with a "tick size" (in which case the actual price is obtained by multiplying the number of ticks by the tick size). In order to be able to handle multiple markets and easily compare their prices, prices can be normalized to fixed point values with a coefficient of 10^8. This makes it possible to encode prices from all known markets without losing any information.

-哈希表62，其用于将字符串变换为整数：整数以便更易于处理。因此，输出总线可以优选使用证券和组ID(标识符)，而不是如同某些市场所做的那样使用名称。为了将名称变换为ID，可以使用包含名称与ID之间的关系的哈希表62。即使当市场发送ID时，也可以增加哈希表以便将市场的ID变换为内部ID。A hash table 62 is used to convert strings into integers for easier processing. Therefore, the output bus can preferably use security and group IDs (identifiers) rather than names, as some markets do. To convert names into IDs, a hash table 62 containing the relationship between names and IDs can be used. Even when the market sends an ID, a hash table can be added to convert the market's ID into an internal ID.

如图3所示，编译器3被配置为，按照诸如Verilog或VHDL的硬件描述语言，处理描述文件5并生成包括有限状态机41(Finite State Machine，也称为“FSM”)的引擎4。下面的描述将参考下列各项做出：包括至少一个XML描述文件的描述文件5，以及由诸如VHDL的硬件描述语言写出并且包括有限状态机41的引擎4。As shown in FIG3 , the compiler 3 is configured to process a description file 5 in accordance with a hardware description language such as Verilog or VHDL and generate an engine 4 including a finite state machine (FSM) 41. The following description will refer to the description file 5 including at least one XML description file and the engine 4 written in a hardware description language such as VHDL and including the finite state machine 41.

描述文件5可以例如为XML(可扩展标记语言的缩写)格式的。根据本发明的实施方案的用于生成解码设备的描述文件5的语法与常规的FAST描述文件的语法相似。更具体而言，描述文件5可以包括对由引擎生成的指令的描述，以及对输入流的格式的描述。The description file 5 can be, for example, in XML (abbreviation of extensible markup language) format. The grammar of the description file 5 for generating a decoding device according to an embodiment of the present invention is similar to the grammar of a conventional FAST description file. More specifically, the description file 5 can include a description of the instructions generated by the engine, and a description of the format of the input stream.

如果输入流源自FAST格式，则描述文件5可以结合从交易所获得的模板的内容。这样的模板可以是XML格式的。则，例如通过使用特定的标记<xi:include>，描述文件5可以参考从市场接收的模板的内容，然后可以补充模板文件以描述解码器设备输出指令。该标记用于在标记的位置结合市场描述文件的内容(模板描述文件内容)。原始的模板描述文件保持不变，使得其可以与可能在流格式演化的情况下创建的未来版本进行比较。If the input stream originates in the FAST format, the description file 5 can incorporate the content of a template received from the market. Such a template can be in XML format. For example, by using the special tag <xi:include>, the description file 5 can reference the content of the template received from the market, and the template file can then be supplemented to describe the decoder device output instructions. This tag is used to incorporate the content of the market description file (the template description file content) at the marked position. The original template description file remains unchanged, allowing it to be compared with future versions that may be created as the stream format evolves.

当输入流是二进制格式时，描述文件不是基于交易所提供的模板的。其可以基于由交易所发布的规范而写成。则该描述文件可以包括对于输入市场数据供给的描述，以及要生成的输出指令的描述。When the input stream is in binary format, the description file is not based on a template provided by the exchange. Instead, it can be written based on the specifications published by the exchange. The description file can then include a description of the input market data feed and a description of the output instructions to be generated.

XML描述文件5的结构相应地取决于特定的市场应用。The structure of the XML description file 5 accordingly depends on the specific market application.

例如，对于使用FAST格式的EUREX流，EUREX市场提供了相应的模板文件。其描述了流的格式、每个消息所使用的“模板”、以及用于每个域的操作符。则，通过使用《xi:include》XML标记，描述文件5将包括模板文件的内容。For example, for EUREX flows using the FAST format, the EUREX market provides a corresponding template file. This file describes the format of the flow, the "template" used for each message, and the operators used for each field. Therefore, by using the <xi:include> XML tag, the description file 5 will include the contents of the template file.

在输入流是二进制格式的NASDAQ流的另一个示例中，不存在NASDAQ提供的、XML或任何其他计算机可读格式的用以描述其流的模板。从而，NASDAQ流的描述和指令完全是在相同的XML文件5中写成的，使得其可读性更强。In another example where the input stream is a NASDAQ stream in binary format, there is no template provided by NASDAQ in XML or any other computer-readable format to describe its stream. Therefore, the description and instructions of the NASDAQ stream are all written in the same XML file 5, making it more readable.

根据本发明写成的描述文件5向现有的FAST模板格式增加了两个特征集：The description file 5 written according to the present invention adds two feature sets to the existing FAST template format:

-第一集被配置为，描述未按FAST编码且可能具有FAST不支持的特征的供给；对于FIX格式的供给，编译器与FIX规格(由QuickFix工程生成)的XML描述兼容。- The first set is configured to describe feeds that are not encoded in FAST and may have features not supported by FAST; for feeds in FIX format, the compiler is compatible with the XML description of the FIX specification (generated by the QuickFix project).

-第二集被配置为，描述解码设备的输出指令；具体而言，当引擎4不能直接地输出完全标准化了的指令时，使得描述文件5适应尽可能接近标准化指令的描述指令。The second set is configured to describe output instructions of the decoding device; specifically, when the engine 4 cannot directly output completely standardized instructions, the description file 5 is adapted to describe instructions that are as close to the standardized instructions as possible.

如此写成的描述文件5能够对扩展了FAST格式的任何数据表示格式的输入流进行解码。通过以额外的标记段补充市场所提供的输入描述文件，能够支持额外的格式并且支持传统FAST格式的演化。The description file 5 written in this way can decode input streams of any data representation format that extends the FAST format. By supplementing the input description file provided by the market with additional marker segments, additional formats can be supported and the evolution of the traditional FAST format can be supported.

XML描述文件5具有比从其生成的VHDL文件41更高水平的抽象性，并且包含更少的代码，这使得修改更容易，而且更不易出现错误。The XML description file 5 has a higher level of abstraction than the VHDL file 41 generated therefrom and contains less code, which makes modification easier and less prone to errors.

根据本发明的一个方面，每个引擎4实例化至少一个令牌解析器40，其被配置为，依据输入流的格式，将输入流打断为有意义的、称为令牌的元素。According to one aspect of the present invention, each engine 4 instantiates at least one tokenizer 40 configured to break the input stream into meaningful elements called tokens according to the format of the input stream.

为了动态生成引擎4，基于令牌解析器40和有限状态机41(其可以适应于全部的格式)，本发明提供了通用代码结构。For the dynamic generation engine 4, the present invention provides a general code structure based on a tokenizer 40 and a finite state machine 41 (which can be adapted to all formats).

图4示出了根据本发明的特定实施方案的令牌解析器40的架构。FIG4 shows the architecture of a tokenizer 40 according to a particular embodiment of the present invention.

令牌解析器40被配置为，接收输入流，对其进行处理，并且输出彼此分开的并且有限状态机41可以使用的令牌。根据本发明的一个方面，令牌可以包括依据市场数据输入流的格式的字节：例如，在二进制格式下，每个令牌对应于单独的字节；在FIX格式下，每个令牌对应于包括字节的FIX域；而在FAST格式下，每个令牌对应于包括字节的FAST域。The tokenizer 40 is configured to receive an input stream, process it, and output separate tokens that can be used by the finite state machine 41. According to one aspect of the present invention, the tokens may comprise bytes in a format according to the market data input stream: for example, in a binary format, each token corresponds to a separate byte; in a FIX format, each token corresponds to a FIX field comprising bytes; and in a FAST format, each token corresponds to a FAST field comprising bytes.

令牌解析器40可以包括：The tokenizer 40 may include:

-对于FAST流，第一类型的令牌解析器，其用于输出在每个停止比特将输入流切断的FAST域。- For FAST streams, a first type tokenizer that outputs FAST fields that cut off the input stream at each stop bit.

-对于FIX流，第二类型的令牌解析器，其用于输出作为分开的令牌的FIX域和域ID，以便能够在读取当前域值的同时读取下一个域ID。第二类型的令牌解析器被布置为，在SOH字符(|)以及＝字符处都切断流。- For FIX streams, a second type of tokenizer for outputting the FIX domain and domain ID as separate tokens so that the next domain ID can be read simultaneously with the current domain value. The second type of tokenizer is arranged to cut the stream at both the SOH character (|) and the = character.

-对于二进制流，第三类型的令牌解析器，其用于输出单独的字节。- For binary streams, a third type of tokenizer, which outputs individual bytes.

在编译最大集合时，根据本发明的实施方案的令牌解析器40不仅具有输出单个令牌的串并转换器(deserializer)功能，还可以输出引擎逻辑的剩余部分所需要的多个令牌。这使得能够在每个时钟周期读取多个令牌。令牌解析器40被配置为，输出有限状态机41可以使用的令牌的阵列(在图4的示例中，3个令牌)。作为返回，有限状态机41被配置为发送其使用的令牌的数量。在下一个时钟周期中，令牌解析器40将读取并在其输出接口表示未使用的令牌以及从输入流获得的新的令牌。When compiling the maximum set, the token parser 40 according to the embodiment of the present invention not only has the deserializer function of outputting a single token, but also can output multiple tokens required by the rest of the engine logic. This makes it possible to read multiple tokens in each clock cycle. The token parser 40 is configured to output an array of tokens that can be used by the finite state machine 41 (in the example of Figure 4, 3 tokens). In return, the finite state machine 41 is configured to send the number of tokens it uses. In the next clock cycle, the token parser 40 will read and represent unused tokens and new tokens obtained from the input stream at its output interface.

如图4所示，令牌解析器40可以包括解析器400，其形成令牌解析器的核心。解析器400被设置为实际将流切断(或分割)为令牌集。As shown in Figure 4, the tokenizer 40 may include a parser 400, which forms the core of the tokenizer. The parser 400 is arranged to actually cut (or split) the stream into sets of tokens.

另外，令牌解析器40可以被配置为，采用输入流的带宽而没有反压(back-pressuring)效果。反压效果一般在核心将其“就绪”信号降低以形成核心连接到其输入等待的时候出现。其副作用为，其降低了核心支持的最大带宽，因为其浪费了时钟周期来等待。在此方面，解析器400可以相应地被配置为，输出可以在来自输入流2的数据的一个字中提供的令牌的最大数量。可以不使用在解析器400的输出处的一些令牌。如果令牌分布在多个字节上，则可以不使用令牌的最大量，而且一些令牌可以从而被标记为无效。例如，对于FAST解析器，如果解析器400的输入处的总线具有8字节的宽度，在编译时限定的令牌的最大量将是8，因为一个令牌至少是一个字节。然而，一些令牌可以是几个字节长。例如，如果2个令牌是两字节长，1个令牌是四字节长，则在解析器的输出可以呈现仅仅3个有效令牌，而其他五个令牌可以被标记为无效。根据另一个示例，如果遇到20字节长的令牌，其可以分布在输入数据的3个字上，而且可能在至少2个时钟周期内没有有效的令牌输出。In addition, the token parser 40 can be configured to use the bandwidth of the input stream without back-pressuring. Back-pressuring generally occurs when the core lowers its "ready" signal to form a wait for the core to connect to its input. As a side effect, it reduces the maximum bandwidth supported by the core because it wastes clock cycles waiting. In this regard, the parser 400 can be configured accordingly to output the maximum number of tokens that can be provided in one word of data from the input stream 2. Some tokens at the output of the parser 400 may not be used. If the tokens are distributed over multiple bytes, the maximum number of tokens may not be used and some tokens may be marked as invalid. For example, for a FAST parser, if the bus at the input of the parser 400 has a width of 8 bytes, the maximum number of tokens defined at compile time will be 8 because a token is at least one byte. However, some tokens may be several bytes long. For example, if 2 tokens are two bytes long and 1 token is four bytes long, only 3 valid tokens may be presented at the output of the parser, while the other five tokens may be marked as invalid. According to another example, if a 20-byte long token is encountered, it may be spread over 3 words of input data, and there may be no valid token output for at least 2 clock cycles.

令牌解析器40可以进一步包括缓冲器401，其用于缓冲令牌，使得以令牌的阵列作为其输出的令牌在每次转移是有效的。则不同于在阵列的一些令牌没有被使用的情况下，在解析器400的输出出现时，阵列的令牌可以被记为有效。这使得设置在缓冲器401下游的读取管理核心402(下文也称为“读取排列核心”)所执行的处理更容易，并且可能实现更高的工作频率。The tokenizer 40 may further include a buffer 401 for buffering tokens so that the tokens outputted as an array of tokens are valid at each transfer. Instead of using some tokens in the array, the tokens in the array can be recorded as valid when the output of the tokenizer 400 appears. This makes the processing performed by the read management core 402 (hereinafter also referred to as the "read alignment core"), which is provided downstream of the buffer 401, easier and may achieve a higher operating frequency.

令牌解析器40的读取排列核心402被设置为使得能够进行部分读取。FSM41可以读取在“读取排列”核心402的输出接口上呈现的全部可用令牌。或者，其可以被配置为，仅读取一部分可用令牌。在这样的实施方案中，在可用令牌中被读取的令牌集可以取决于条件，尤其是设定了将要读取的令牌的数量的条件。例如，将要读取的令牌的数量可以取决于特定令牌的值。在有限状态机41未就绪时，读取排列核心402的操作具有特定的益处。例如，如果将要读取的令牌的数量取决于正被读取的特定令牌的值，则读取该特定令牌，然后，依据其值，在接下来的一个或多个周期中将会读取更多或更少的令牌。将要读取的令牌的数量也可以取决于关于有限状态机41的结构的某些参数。The read permutation core 402 of the token parser 40 is arranged to enable partial reads. The FSM 41 can read all available tokens presented on the output interface of the "read permutation" core 402. Alternatively, it can be configured to read only a portion of the available tokens. In such an embodiment, the set of tokens that are read from the available tokens can depend on a condition, in particular a condition that sets the number of tokens to be read. For example, the number of tokens to be read can depend on the value of a particular token. The operation of the read permutation core 402 when the finite state machine 41 is not ready has particular benefits. For example, if the number of tokens to be read depends on the value of a particular token being read, then that particular token is read and then, depending on its value, more or fewer tokens will be read in the next one or more cycles. The number of tokens to be read can also depend on certain parameters about the structure of the finite state machine 41.

读取的令牌数量被发送回“读取排列”核心402，使得未被读取的剩余令牌可以附加到新的令牌并且在接下来的一个或多个时钟周期中再次呈现在其输出接口。The number of tokens read is sent back to the "read queue" core 402 so that the remaining tokens that have not been read can be appended to new tokens and presented again at its output interface in the next one or more clock cycles.

对于二进制流，令牌解析器40可以不包括解析器400。此外，可以使用特殊版本的读取排列核心402，以使得在流中能够读取随机字节长度，输入流的每个字已经包含了令牌的最大数量，因为令牌实际是流中的单独的字节。实际上，对于二进制流，令牌对应于字节，从而不需要解析器来将流分割为字节然后在缓冲器中对所获得的字节进行组装。For binary streams, tokenizer 40 may not include parser 400. In addition, a special version of read permutation core 402 may be used to enable reading of random byte lengths in the stream, with each word of the input stream already containing the maximum number of tokens, since tokens are actually individual bytes in the stream. In fact, for binary streams, tokens correspond to bytes, eliminating the need for a parser to split the stream into bytes and then assemble the resulting bytes in a buffer.

对于FIX流，解析器400可以创建上至输入字节的数量的一半，因为每个令牌消耗一个界定字符(“|”或“＝”)。For a FIX stream, the parser 400 may create up to half the number of input bytes because each token consumes one delimiting character ("|" or "=").

对于FAST流，解析器400可以创建与输入字节一样多的FAST令牌，因为其中的每一个可以包含停止字符。For a FAST stream, the parser 400 may create as many FAST tokens as there are input bytes, since each of them may contain a stop character.

由原始提取的域，如此获得的令牌可以包括：关于域的类似其长度的信息，以及其转换为各种格式的值(例如，对于FIX，ASCII字符串的二进制表示，或者对于FAST，域的右侧填充(right-padded)版本)。From the originally extracted fields, the tokens thus obtained may include information about the field like its length, and its value converted to various formats (e.g., a binary representation of the ASCII string for FIX, or a right-padded version of the field for FAST).

根据本发明的实施方案，令牌解析器40的操作受到有限状态机41的控制。According to an embodiment of the present invention, the operation of the tokenizer 40 is controlled by a finite state machine 41 .

具体而言，有限状态机41被配置为，执行下列动作中的至少一些：Specifically, the finite state machine 41 is configured to perform at least some of the following actions:

-从令牌解析器40读取所选择的令牌/域的数量，- read the number of selected tokens/domains from the tokenizer 40,

-将读取的域复制到存储元件，- copy the read field to the storage element,

-对于FAST流应用所选择的操作符，和/或- Apply the selected operator to the FAST stream, and/or

-在确定的时刻发起输出指令。-Initiate output instructions at a certain time.

从XML描述文件5动态地生成的有限状态机41形成引擎4的核心。状态机41的结构对应于所解码的供给的结构，具有在每个消息或消息选择部分的分支，以及用于重复消息内部的元素的循环。A finite state machine 41 dynamically generated from an XML description file 5 forms the core of the engine 4. The structure of the state machine 41 corresponds to the structure of the decoded feed, with branches at each message or message selection, and loops for repeating elements inside a message.

有限状态机41使得能够在每个状态读取尽可能多的令牌。然而，某些域的存在以及要读取的令牌的数量可能取决于其前面的域的值或存在。The finite state machine 41 enables as many tokens as possible to be read in each state. However, the presence of certain fields and the number of tokens to be read may depend on the value or presence of the preceding field.

状态机41还可以被配置为，处理在对包进行解码时可能出现的任何错误，例如残缺的消息、意外的包的结束、或者未知的消息。State machine 41 may also be configured to handle any errors that may occur while decoding the packet, such as a malformed message, an unexpected end of packet, or an unknown message.

由于状态机从XML描述文件生成，每个状态机41取决于市场数据输入流，并且可以在每次输入流格式更新时更新。每个状态机41可以具有大量的状态。例如，在期权和资产净值市场应用中，由交易所提供的FAST消息模板可能非常大，则有限状态机41可以包括多于600个状态。Because the state machines are generated from XML description files, each state machine 41 depends on the market data input stream and can be updated every time the input stream format is updated. Each state machine 41 can have a large number of states. For example, in options and net asset value market applications, the FAST message templates provided by the exchange can be very large, and the finite state machine 41 can include more than 600 states.

根据本发明的另一个方面，引擎4可以包括存储元件7的集合，其用于存储在输入流中读取的数据，并且随后根据包括在描述文件5中的信息而输出包含该数据的指令。具体而言，存储元件7可以包括内部寄存器。仅出于示意的目的，下面的描述将参考实施为内部寄存器的存储元件7而做出。According to another aspect of the present invention, the engine 4 may include a set of storage elements 7 for storing data read in the input stream and subsequently outputting instructions containing the data based on information included in the description file 5. Specifically, the storage elements 7 may include internal registers. For illustrative purposes only, the following description will be made with reference to the storage elements 7 being implemented as internal registers.

包括在输入流2中的信息取决于市场。对于二进制市场，输入流一般仅具有关键信息，从而来自输入流的多数域被存储并且在输出指令中被发送。在FAST或FIX市场中，输入流一般地包括许多信息。可以仅存储和在输出指令中发送关键信息(即，被确定为目标客户感兴趣的信息)。The information included in the input stream 2 depends on the market. For binary markets, the input stream typically contains only critical information, so most fields from the input stream are stored and sent in the output instructions. In FAST or FIX markets, the input stream typically contains a lot of information. Only critical information (i.e., information determined to be of interest to the target customer) can be stored and sent in the output instructions.

为了从发起输出指令的状态机41取得信号，设置了分开的且计时的过程。该过程则通过选择内部寄存器而执行这些指令，并且将这些指令复制到输出指令端口。具体而言，有限状态机41可以生成称为《指令发送》的信号以及称为《指令数量》的进一步的信号。在指令中复制的寄存器可以依据指令数量来选择。待发送的指令的描述由描述文件提供。利用分开，计时的过程在状态机41与输出总线之间、在选择寄存器以复制到输出总线的多路调制器之后增加寄存器级，这使得时序的截止更容易。In order to obtain a signal from the state machine 41 that initiates the output instruction, a separate and timed process is provided. This process executes these instructions by selecting internal registers and copies these instructions to the output instruction port. Specifically, the finite state machine 41 can generate a signal called "Instruction Sent" and a further signal called "Number of Instructions". The registers copied in the instruction can be selected based on the number of instructions. The description of the instruction to be sent is provided by a description file. Using a separate, timed process, a register stage is added between the state machine 41 and the output bus, after the multiplexer that selects the register to copy to the output bus, which makes it easier to cut off the timing.

根据本发明的解码设备10可以进一步包括由引擎编译器生成的静态计数器(也称为静态寄存器)。静态计数器可以基于状态机41所提供的信号而更新。静态计数器可以基于从状态机41取得的信号而增长或设定。其可以从诸如Avalon MM接口的标准接口读取。保存在静态计数器中的信息可以用于调试和检测错误配置。一些寄存器由编译器自动添加(对错误进行解码的计数器、意外包的结束的计数器)。其他寄存器可以在描述文件5中得到描述。为了能够监控用于不同的消息类型的消息计数，计数器可以对于每种消息类型而添加。其可以尤其用于检查测试向量是否包含足够数量的每种类型的消息。The decoding device 10 according to the present invention may further include a static counter (also referred to as a static register) generated by the engine compiler. The static counter can be updated based on a signal provided by the state machine 41. The static counter can be increased or set based on a signal obtained from the state machine 41. It can be read from a standard interface such as the Avalon MM interface. The information stored in the static counter can be used for debugging and detecting misconfigurations. Some registers are automatically added by the compiler (counters for decoding errors, counters for the end of unexpected packets). Other registers can be described in the description file 5. In order to be able to monitor message counts for different message types, a counter can be added for each message type. It can be used in particular to check whether the test vector contains a sufficient number of messages of each type.

图5是示出根据本发明的特定实施方案的解码方法的流程图。FIG5 is a flow chart illustrating a decoding method according to a specific embodiment of the present invention.

在步骤500，在有限状态机的当前状态下，接收给定格式的、已分割为令牌的输入流2，以及累积在内部寄存器中。如果输入流是按FAST格式或源自FAST的格式编码的，则令牌可以包括FAST域；如果输入流是按FIX编码的，则令牌可以包括FIX域；或者令牌可以包括字节(二进制格式)。In step 500, in the current state of the finite state machine, an input stream 2 of a given format, which has been segmented into tokens, is received and accumulated in an internal register. If the input stream is encoded in FAST format or a format derived from FAST, the token may include a FAST field; if the input stream is encoded in FIX format, the token may include a FIX field; or the token may include bytes (binary format).

在步骤502，确定是否已经接收到了足够的令牌。在该步骤，对在上一步接收到的令牌(对于二进制输入流，字节；对于基于FIX的输入流，FIX域；对于基于FAST的输入流，FAST域)的数量与对应于有限状态机41的当前状态下所期望接收到的令牌(字节/域)的数量的阈值数量进行比较。依据描述文件5的内容，阈值数量可以是由编译器在编译时固定的。In step 502, it is determined whether sufficient tokens have been received. In this step, the number of tokens (bytes for binary input streams, FIX fields for FIX-based input streams, and FAST fields for FAST-based input streams) received in the previous step is compared with a threshold number corresponding to the number of tokens (bytes/fields) expected to be received in the current state of finite state machine 41. Depending on the content of description file 5, the threshold number may be fixed by the compiler at compile time.

如果令牌的数量足够，则引擎前进至其他步骤503至505。否则，该过程保持在相同的FSM状态(506)并且等待下一个时钟周期。在下一个时钟周期，将重复步骤500和502以确定是否可以读取更多的令牌。If the number of tokens is sufficient, the engine proceeds to other steps 503 to 505. Otherwise, the process remains in the same FSM state (506) and waits for the next clock cycle. In the next clock cycle, steps 500 and 502 will be repeated to determine whether more tokens can be read.

在步骤503，生成输出指令。In step 503, an output instruction is generated.

依据XML描述文件，有限状态机41的一些状态在引擎4的输出接口生成指令。这些指令可以包括第一步骤500期间或者之前的周期期间(其他FSM状态下的之前的步骤500的迭代)读取的数据。According to the XML description file, some states of the finite state machine 41 generate instructions at the output interface of the engine 4. These instructions may include data read during the first step 500 or during previous cycles (previous iterations of step 500 in other FSM states).

在步骤504，可以进一步检查某些由于输入流的格式导致的错误。这些错误可能由于被截断的包所导致的意外的包的结束而出现，或者在超过了记录时出现。记录指的是长度在输入流中给出的域集。当超过记录时，例如，或者是输入流提供的记录长度是错的，或者是由于例如用于生成引擎的描述文件5中的错误而读取了太多数据。In step 504, further checks may be performed for errors due to the format of the input stream. These errors may occur due to unexpected end-of-packet situations caused by truncated packets, or when records are exceeded. A record is a set of fields whose lengths are given in the input stream. When records are exceeded, for example, either the record length provided by the input stream is incorrect, or too much data was read due to errors in, for example, the description file 5 used to generate the engine.

在步骤505，选择有限状态机41的下一个状态，该过程跳转到该状态。有限状态机41的下一个状态可以选自：In step 505, the next state of the finite state machine 41 is selected, and the process jumps to this state. The next state of the finite state machine 41 can be selected from:

-在第一步骤500中或者之前的周期中(其他FSM状态下的之前的步骤500的迭代)读取的数据；- data read in the first step 500 or in a previous cycle (a previous iteration of step 500 in another FSM state);

-在步骤504获得的错误检查结果(如果检测到了错误，则有限状态机跳转到错误指定状态，在该状态下，有限状态机在正常处理下一个包之前等待包的结束)；或者- the error check result obtained in step 504 (if an error is detected, the finite state machine jumps to the error-specified state, in which the finite state machine waits for the end of the packet before processing the next packet normally); or

-反压信号，其来自表示链中的下一个核心的信号，尤其是来自可以请求引擎降速以便引擎移动到其不活动的特殊休息状态的订单管理核心12的信号。- A backpressure signal, which is a signal from the next core in the chain, in particular from the order management core 12 that can request an engine to be spun down so that it moves to its special rest state where it is inactive.

应当注意，全部步骤500至506在单独的时钟周期中执行，而且在每个新的时钟周期，所有这些步骤重复。It should be noted that all of steps 500 to 506 are performed in a single clock cycle, and that at each new clock cycle, all of these steps are repeated.

本领域技术人员易于理解，图5中的一些步骤可以根据其他顺序执行。例如，可以在生成指令输出(步骤503)之前检查错误(步骤504)。Those skilled in the art will readily appreciate that some steps in Figure 5 may be performed in other orders. For example, errors may be checked (step 504) before generating the instruction output (step 503).

图6是示出根据本发明的简化了的示例的有限状态机的操作的流程图。在图6的示例中，从市场接收的数据流是二进制格式的，每个接收到的包包括头部，其之后是唯一的消息(在三个可能的消息A、B和C之中)。可以使用的不同的消息和头部具有下列特征：FIG6 is a flow chart illustrating the operation of a finite state machine according to a simplified example of the present invention. In the example of FIG6 , the data stream received from the market is in binary format, and each received packet includes a header followed by a unique message (among three possible messages A, B, and C). The different messages and headers that can be used have the following characteristics:

-头部包括：-Head includes:

4字节的第一域；4-byte first field;

2字节的第二域；2-byte second field;

1字节的消息类型；1 byte message type;

消息A包括：Message A includes:

4字节的第一域A1；4-byte first field A1;

2字节的第二域A2。The second field A2 is 2 bytes.

-消息B包括：-Message B includes:

3字节的第一域B1；3-byte first field B1;

1字节的第二域B2。The second field B2 is 1 byte.

-消息C包括：-Message C includes:

4字节的第一域C1；以及A 4-byte first field C1; and

对于若干子消息(对应于1字节)：For several sub-messages (corresponding to 1 byte):

1字节的第二域C2；1-byte second field C2;

2字节的第三域C3。The third field C3 is 2 bytes.

域C2和C3重复的次数为子消息的数量。Fields C2 and C3 are repeated as many times as the number of sub-messages.

在图6的示例中，连同每个消息A和B，在消息A或B(其包括包含在消息中的信息以及头部)的末端发送指令。对于消息C，发送相似的指令。然而，指令是随着子消息发送的，以便传送包括在子消息中的全部信息。In the example of FIG6 , instructions are sent at the end of message A or B (which includes the information contained in the message and a header) along with each of messages A and B. Similar instructions are sent for message C. However, the instructions are sent with the sub-message so that all the information included in the sub-message is transmitted.

有限状态机41被配置为，存储从通过解码设备10在内部寄存器中获得的输入流信息的至少一部分，并且在输出指令中发送该信息的至少一部分。The finite state machine 41 is configured to store at least a portion of the input stream information obtained from the decoding device 10 in an internal register and to transmit at least a portion of the information in an output instruction.

图6示出了为4字节输入总线生成的有限状态机41。如图所示，在特定状态(2、A2、C2、C3)下，没有读取最大数量的字节(在示例中，4字节)：Figure 6 shows a finite state machine 41 generated for a 4-byte input bus. As shown, in a particular state (2, A2, C2, C3), the maximum number of bytes (in this example, 4 bytes) is not read:

-在状态A2，这因为达到了包的结束而发生；- in state A2, this occurs because the end of the packet is reached;

-在状态2，这因为需要在状态3做出的决策而发生；可以考虑在状态4读取第四字节。在这样的情况下，应当决定使用哪个内部寄存器(A1、B1、C1等)。如果第四字节没有存储在任何寄存器中，其不能再被使用。- In state 2, this happens because of the decision that needs to be made in state 3; it is possible to consider reading the fourth byte in state 4. In this case, it should be decided which internal register to use (A1, B1, C1, etc.). If the fourth byte is not stored in any register, it can no longer be used.

-在状态C2和C3，这是由于循环的存在以及由于子消息的数量可以等于零；- in states C2 and C3, this is due to the existence of loops and because the number of submessages can be equal to zero;

-在状态3和4，没有数据被读取。- In states 3 and 4, no data is read.

如同本领域技术人员易于理解的，图6仅是为了方便理解本发明的特定实施方案的简化的示例。本发明不限于参考图6描述的包、消息、头部、域的示例性结构。具体而言，由市场提供的输入数据流可以包括多个消息类型、每个包的多个消息、具有包的大小和消息的大小并且在接收到包/消息时进行检查的头部等。此外，尽管在图6的示意性表示中，没有表示有限状态机41的错误和反压处理状态，但是，本领域技术人员易于理解，可以包括这些状态。As will be readily appreciated by those skilled in the art, FIG6 is merely a simplified example to facilitate understanding of a specific embodiment of the present invention. The present invention is not limited to the exemplary structure of packets, messages, headers, and fields described with reference to FIG6 . Specifically, the input data stream provided by the market may include multiple message types, multiple messages per packet, a header having a packet size and a message size that is checked upon receipt of a packet/message, and the like. Furthermore, although the error and backpressure handling states of finite state machine 41 are not shown in the schematic representation of FIG6 , it will be readily appreciated by those skilled in the art that these states may be included.

对于FAST格式的市场数据输入流，对每个域应用操作符。此外，FAST输入流的某些域可以是“存在映射”域，特定状态下读取的令牌的数量取决于该“存在映射”域。在图6的示例中，在给定状态下读取的令牌的数量是预定的：例如在状态A2，读取两个令牌(＝2字节)。在FAST，读取的令牌的数量可以进一步取决于存在映射。For a FAST-formatted market data input stream, operators are applied to each field. Furthermore, certain fields of the FAST input stream may be "presence-mapped" fields, and the number of tokens read in a particular state depends on the presence-mapped field. In the example of FIG6 , the number of tokens read in a given state is predetermined: for example, in state A2 , two tokens (= 2 bytes) are read. In FAST, the number of tokens read can further depend on the presence-mapped field.

另外，对于FAST格式的市场数据输入流，可以对不同的读取的域中的每一个并行的应用操作符。Additionally, for a FAST format market data input stream, operators can be applied in parallel to each of the different read fields.

应当注意，特定的编码可以用于FAST中的十进制小数：当流中待传递的值是可选的时，可以使用唯一且为空的令牌来指示该值不存在，非空令牌则被解释为十进制小数的指数，而有限状态机41则读取进一步的令牌以用于十进制小数的尾数(该值被认为存在)。这是另一个示例，其中，读取的令牌的数量可以取决于其他令牌的值。It should be noted that a special encoding can be used for decimal numbers in FAST: when the value to be passed in the stream is optional, a single empty token can be used to indicate that the value is not present, a non-empty token is interpreted as the exponent of the decimal number, and the finite state machine 41 reads a further token for the mantissa of the decimal number (which is considered to be present). This is another example where the number of tokens read can depend on the values of other tokens.

根据本发明的解码设备尤其适应于在可重构平台上实施，例如FPGA。The decoding device according to the present invention is particularly suitable for implementation on a reconfigurable platform, such as an FPGA.

图7显示了市场数据处理系统100的示例性实施方式，其位于标准服务器(主机系统)71内部的PCIe主板70上的FPGA中。图1表示的链(网络/UDP获取2、解码设备10、额度聚类和账簿建立单元13、消息分发和传输单元14)可以通过下列各项补充：Figure 7 shows an exemplary embodiment of a market data processing system 100, which is located in an FPGA on a PCIe motherboard 70 within a standard server (host system) 71. The chain shown in Figure 1 (network/UDP acquisition 2, decoding device 10, quota clustering and ledger creation unit 13, message distribution and transmission unit 14) can be supplemented by the following:

-UDP和以太网堆栈700，其被配置为在标准以太网上发送输出消息。为了优化性能，该核心可以通过以太网连接(该以太网连接与所述网络上的客户端一样多)连接至客户端网络。在相同网络上的其他计算机系统72中的客户端应用可以以软件实施、由硬件加速或者可替选地完全由硬件实施。或者，客户端应用可以直接地连接至系统的输出端口，以便节省网络装置所增加的延迟。- UDP and Ethernet stack 700, which is configured to send outgoing messages over standard Ethernet. To optimize performance, the core can be connected to the client network via Ethernet connections (there are as many Ethernet connections as there are clients on the network). Client applications in other computer systems 72 on the same network can be implemented in software, accelerated by hardware, or alternatively implemented entirely in hardware. Alternatively, client applications can be connected directly to the system's output ports to save on the latency added by the network devices.

-PCIe互联端点701，其被配置为连接至主机系统。- A PCIe interconnect endpoint 701 configured to connect to a host system.

-DMA(“Direct Memory Access，直接存储器访问”)702，其被配置为经由主板的PCIe连接而向主机系统RAM 73发送输出消息；客户端应用从而可以在主机系统的CPU 707上运行，并且接受由DMA发送的消息。- DMA ("Direct Memory Access") 702, which is configured to send outgoing messages to the host system RAM 73 via the motherboard's PCIe connection; a client application can thus run on the host system's CPU 707 and accept messages sent by the DMA.

-第二“NIC”DMA 704(“NIC”是网络接口卡的缩写)，其被配置为，通过通常由市场数据处理系统的硬件使用的以太网端口，从主机系统的软件接收/向主机系统的软件发送以太网包。PCIe主板70从而可以被操作系统视作普通的网络接口卡。这对于各种实际的目的(例如，发送多路群组订阅、登陆到市场交易的系统、能够检查连接性)可能是需要的。在特定的实施方案中，重要的并且对于延迟敏感的数据可以由硬件直接地处理，而其他数据可以由软件处理。这样的配置还使得能够直接连接至交易网络1，而无需通过网络交换机或路由器的额外跃点。- A second "NIC" DMA 704 ("NIC" stands for Network Interface Card), which is configured to receive and send Ethernet packets from/to the host system's software via the Ethernet port typically used by the market data processing system's hardware. The PCIe motherboard 70 can thus be viewed by the operating system as an ordinary network interface card. This may be desirable for various practical purposes (e.g., sending multiple group subscriptions, logging into the market trading system, and being able to check connectivity). In certain embodiments, important and latency-sensitive data can be processed directly by the hardware, while other data can be processed by the software. Such a configuration also enables direct connection to the trading network 1 without requiring additional hops through network switches or routers.

-配置和监控逻辑706，其用于配置其他核心，包括例如订阅至金融证券更新，以及通过PCIe连接来监控系统的状态和性能。为利用该配置和监控逻辑，可以通过软件API提供以在主机CPU 707上执行的软件。- Configuration and monitoring logic 706, which is used to configure other cores, including, for example, subscribing to financial securities updates, and monitoring the status and performance of the system via the PCIe connection. To utilize this configuration and monitoring logic, software executing on the host CPU 707 may be provided via a software API.

在FPGA，PCIe主板70提供可以被市场数据处理系统100用作“外部存储器”的存储器芯片，以及各种支持功能，例如电源调节，以便以主板上部件所需的各种电压向这些部件供电。其还可以包括物理连接器，其用于多重以太网连接。In the FPGA, PCIe motherboard 70 provides memory chips that can be used as "external memory" by the market data processing system 100, as well as various supporting functions, such as power regulation to power the components on the motherboard at the various voltages required by these components. It may also include physical connectors for multiple Ethernet connections.

这样的主板中的几个可以插入主机系统的PCIe插槽。这使得能够度量处理能力，以便例如支持更多的市场交易。主板可以经由主机的PCIe互联而彼此通信。Several of these motherboards can be plugged into a host system's PCIe slots. This enables scaling of processing power, for example to support more market transactions. The motherboards can communicate with each other via the host's PCIe interconnect.

主机系统71可以是标准的计算机服务器。CPU 707可以被选择为具有指向承载市场数据处理系统的PCIe主板的直接PCIe连接，以便最大化其上运行的客户端应用的性能。主机系统还可以装备有网络接口708，其用于连接至配置和管控网络709。这使得能够具有用于配置和管控服务器的专用网络。该服务器还可以装备有任何的标准部件，例如大容量存储设备(硬盘)。The host system 71 can be a standard computer server. The CPU 707 can be selected to have a direct PCIe connection to the PCIe motherboard hosting the market data processing system to maximize the performance of the client applications running on it. The host system can also be equipped with a network interface 708 for connecting to a configuration and management network 709. This enables a dedicated network for configuring and managing the server. The server can also be equipped with any standard components, such as a mass storage device (hard disk).

将市场交易网络、客户端网络以及配置和管控网络在物理上分开具有安全性以及性能上的益处。Physically separating the marketplace network, client network, and configuration and management network has security and performance benefits.

即使本发明不限于使用PCIe作为FPGA到主机系统的接口，PCIe由于特定的益处而是当前使用最广泛的用于在服务器与外插卡之间的高速传送的总线。然而，本领域技术人员易于理解，市场数据处理系统100可以以任何其他的总线作为接口。Although the present invention is not limited to using PCIe as the interface between the FPGA and the host system, PCIe is currently the most widely used bus for high-speed transmission between servers and add-in cards due to its specific advantages. However, those skilled in the art will readily appreciate that the market data processing system 100 can use any other bus as an interface.

本发明基于代码生成方式，还允许对以任何数据表示格式提供的输入流的并行解码(引擎4建立在有限状态机41周围，由编译器3从描述文件5建立，并且由解码设备10实例化)。The invention is based on a code generation approach and also allows parallel decoding of input streams provided in any data representation format (the engine 4 is built around a finite state machine 41, created by the compiler 3 from a description file 5 and instantiated by the decoding device 10).

这使得能够符合金融市场的性能需求，同时易于适应于新的输入流格式。对于被供给以解码的每个市场数据，根据本发明的实施方案的解码器维持10Gb/s的数据解码，而没有反压施加到网络上。This enables compliance with the performance demands of financial markets while being easily adaptable to new input stream formats.For each market data supplied for decoding, a decoder according to an embodiment of the present invention maintains 10 Gb/s of data decoding with no back pressure applied to the network.

解码设备10易于与标准化的到用户逻辑的输出总线使用，所述用户逻辑包括依据供给的特性的特定于市场的部分。用于交易的共用的市场数据域中的多数符合输出总线的标准化部分，使得用户逻辑能够支持不同的市场数据供给而无需改变设计。The decoding device 10 is easy to use with a standardized output bus to user logic, which includes market-specific portions depending on the characteristics of the feed. Most of the common market data fields used for trading conform to the standardized portion of the output bus, enabling the user logic to support different market data feeds without requiring design changes.

根据本发明，可以并行处理令牌，同时解码设备的整体过程是顺序的。通过使几个令牌能够平行处理，本发明提高了解码设备10的性能。According to the present invention, tokens can be processed in parallel, while the overall process of the decoding device is sequential. By enabling several tokens to be processed in parallel, the present invention improves the performance of the decoding device 10.

在本发明的特定实施方案中，解码设备10可以实施为由各自的FPGA逻辑执行的多个解码器的形式，尤其是两个解码器，以便并行处理市场接收的输入流。在这样的实施方案中，每个解码器可以包括其自己的输出格式化单元以及各自的令牌解析器40。这种实施方案可以在特定情况下应用，例如，在解码器连接至不同的10G端口时，以及在其每个仅处理10G和/或当同时解码来自不同市场的不同格式的几个市场数据流时。此外，在种实施方案中，可以在解码器与订单管理核心之间使用判定器(arbiter)设备，使得订单管理核心不会接收到两次指令。In a specific embodiment of the present invention, the decoding device 10 can be implemented in the form of multiple decoders, in particular two decoders, each executed by its own FPGA logic, in order to process the input streams received by the market in parallel. In such an embodiment, each decoder can include its own output formatting unit and its own tokenizer 40. This embodiment can be applied in specific cases, for example, when the decoders are connected to different 10G ports, each of which processes only 10G and/or when decoding several market data streams of different formats from different markets at the same time. In addition, in such an embodiment, an arbiter device can be used between the decoder and the order management core so that the order management core does not receive instructions twice.

Claims (15)

Translated fromChinese

1.一种解码设备(10)，其实施在集成电路上，用于解码以给定的数据表示格式所接收的市场数据输入流，所述解码设备包括：1. A decoding device (10) implemented on an integrated circuit for decoding a market data input stream received in a given data representation format, the decoding device comprising:建立在有限状态机(41)周围的引擎(4)，引擎(4)从至少一个描述文件(5)生成，并且利用至少一个令牌解析器配置为在每个时钟周期期间并行地将输入市场数据流分割为多个令牌并且读取令牌集，an engine (4) built around a finite state machine (41), the engine (4) generated from at least one description file (5) and configured with at least one tokenizer to split an input market data stream into a plurality of tokens and read the token set in parallel during each clock cycle,所述引擎包括用于累积读取的所述令牌集的存储元件集，The engine comprises a set of storage elements for accumulating the set of tokens read,所述引擎配置为根据从所述描述文件获取的涉及在存储元件中累积的令牌的条件，从在所述存储元件中累积的令牌生成输出指令，以及The engine is configured to generate output instructions from tokens accumulated in the storage element according to conditions obtained from the description file concerning the tokens accumulated in the storage element, and所述引擎配置为基于从所述描述文件获取的触发条件选择有限状态机的下一个状态，其中，所述引擎在有限状态机的当前状态下操作。The engine is configured to select a next state of a finite state machine based on a trigger condition obtained from the description file, wherein the engine operates in a current state of the finite state machine.2.根据权利要求1所述的解码设备，其中，如果满足涉及在存储元件中累积的令牌的条件，则引擎在相同的时钟周期中执行分割、累积、输出指令生成和下一个状态选择操作。2. The decoding device of claim 1, wherein if a condition involving tokens accumulated in a storage element is satisfied, the engine performs splitting, accumulation, output instruction generation, and next state selection operations in the same clock cycle.3.根据权利要求1所述的解码设备，其中，引擎配置为在连续的时钟周期期间保持在存储元件中累积令牌，直到满足涉及在存储元件中累积的令牌的条件。3 . The decoding device of claim 1 , wherein the engine is configured to keep accumulating tokens in the storage element during successive clock cycles until a condition related to the tokens accumulated in the storage element is satisfied.4.根据权利要求1所述的解码设备，其中，所述条件涉及在存储元件中累积的令牌的数量。The decoding device according to claim 1 , wherein the condition relates to the number of tokens accumulated in the storage element.5.根据权利要求1所述的解码设备，其中，所述至少一个令牌解析器(40)受控于有限状态机(41)，有限状态机控制在每个时钟周期期间读取的令牌的数量。5. The decoding device according to claim 1, wherein the at least one tokenizer (40) is controlled by a finite state machine (41) which controls the number of tokens read during each clock cycle.6.根据权利要求5所述的解码设备，其中，所述至少一个令牌解析器(40)包括解析器(400)和缓冲器(401)，所述解析器(400)用于依据输入流的数据表示格式而将输入流(1)分割为令牌，所述缓冲器(401)用于缓冲由解析器(400)提供的令牌。6. The decoding device according to claim 5, wherein the at least one token parser (40) comprises a parser (400) and a buffer (401), wherein the parser (400) is used to split the input stream (1) into tokens according to the data representation format of the input stream, and the buffer (401) is used to buffer the tokens provided by the parser (400).7.根据权利要求5和6中的任一项所述的解码设备，其中，所述至少一个令牌解析器(40)包括读取管理核心(402)，其用以读取从所述输入流的分割获得的令牌集，并且在其输出接口呈现读取的令牌。7. A decoding device according to any one of claims 5 and 6, wherein the at least one token parser (40) includes a read management core (402) which is used to read the set of tokens obtained from the segmentation of the input stream and present the read tokens at its output interface.8.根据权利要求7所述的解码设备，其中，在特定状态下将要读取的令牌集是基于在每个时钟周期中将要读取的令牌的数量的条件所确定的。8 . The decoding device according to claim 7 , wherein the set of tokens to be read in a specific state is determined based on a condition of the number of tokens to be read in each clock cycle.9.根据权利要求8所述的解码设备，其中，所述将要读取的令牌的数量的条件取决于特定令牌的值，有限状态机(41)被配置为：9. The decoding device according to claim 8, wherein the condition of the number of tokens to be read depends on the value of a specific token, the finite state machine (41) being configured to:-在当前时钟周期读取所述特定令牌的值，并且- read the value of said particular token during the current clock cycle, and-在下一个时钟周期，依据所述特定令牌的读取值，读取可用令牌中的令牌的数量。- In the next clock cycle, according to the read value of the specific token, read the number of tokens in the available tokens.10.根据权利要求7所述的解码设备，其中，将尚未被有限状态机从读取管理核心读取的令牌附加到新的令牌，并且在接下来的一个或多个时钟周期内将令牌再次呈现在所述读取管理核心(402)的输出接口。10. The decoding device according to claim 7, wherein the token that has not yet been read by the finite state machine from the read management core is appended to the new token, and the token is presented again at the output interface of the read management core (402) within the next one or more clock cycles.11.根据权利要求10所述的解码设备，其中，输入流具有二进制格式，并且读取管理核心(402)被配置为，允许输入流中的随机字节长度的读取。11. The decoding device of claim 10, wherein the input stream has a binary format and the read management core (402) is configured to allow reading of random byte lengths in the input stream.12.根据权利要求1所述的解码设备，其中，所述触发条件包括下列条件中的至少一个：涉及在所述有限状态机的当前状态下执行的用于确定输入数据流是否包括格式化错误的错误检测结果的条件、依据存储在存储元件中的数据的条件、以及依据从处理线程中的下一个核心接收的反压信号的条件。12. The decoding device of claim 1 , wherein the trigger condition comprises at least one of: a condition involving an error detection result executed in a current state of the finite state machine for determining whether an input data stream includes a formatting error, a condition based on data stored in a storage element, and a condition based on a backpressure signal received from a next core in a processing thread.13.根据权利要求1所述的解码设备，其中，所述输入流的格式是基于FIX的、基于FAST的或者二进制格式的。13 . The decoding device according to claim 1 , wherein a format of the input stream is a FIX-based, a FAST-based, or a binary format.14.根据权利要求1所述的解码设备，其中，该解码设备包括转换单元(6)集，其用以进一步标准化由引擎(4)输出的指令。14. The decoding device according to claim 1, wherein the decoding device comprises a set of conversion units (6) for further standardizing the instructions output by the engine (4).15.一种用于解码输入市场数据流的方法，其以给定的数据表示格式接收，所述方法实施在集成电路上，其中，对于每个接收到的市场数据流，所述方法包括提供从至少一个描述文件(5)生成的有限状态机，所述方法进一步包括下列步骤：15. A method for decoding an input market data stream received in a given data representation format, the method being implemented on an integrated circuit, wherein for each received market data stream, the method comprises providing a finite state machine generated from at least one description file (5), the method further comprising the steps of:i)在每个时钟周期期间并行地将市场数据输入流分割为多个令牌并且读取令牌集，i) splitting the market data input stream into multiple tokens and reading the token set in parallel during each clock cycle,ii)在存储元件中累积所读取的所述令牌集，ii) accumulating said set of tokens read in a storage element,iii)根据从所述描述文件获取的涉及在存储元件中累积的令牌的条件，从在所述存储元件中累积的令牌生成输出指令，以及iii) generating output instructions from the tokens accumulated in the storage element according to conditions obtained from the description file relating to the tokens accumulated in the storage element, andiv)基于从所述描述文件获取的触发条件选择有限状态机的下一个状态，iv) selecting the next state of the finite state machine based on a trigger condition obtained from the description file,其中，在有限状态机的当前状态下执行步骤i)至iv)。Here, steps i) to iv) are performed in the current state of the finite state machine.