RELATED APPLICATIONThe present application is related to commonly assigned and co-pending U.S. patent application Ser. No. 10/897,888 (Attorney Docket No. H0003944-5802) entitled “RECONFIGURABLE COMPUTING ARCHITECTURE FOR SPACE APPLICATIONS,” filed on Jul. 23, 2004, which is incorporated herein by reference, and also referred to here as the '888 Application.
BACKGROUNDIn one type of space application, a device traveling in space transmits data to a device located on Earth. A device traveling in space is also referred to here as a “space device.” Examples of space devices include, without limitation, a satellite and a space vehicle. A device located on Earth is also referred to here as an “Earth-bound device.” An example of an Earth-bound device is a mission control center. Data that is transmitted from a space device to an Earth-bound device is also referred to here as “downstream data” or “payload data.” Examples of payload data include, without limitation, scientific data obtained from one or more sensors or other scientific instruments included in or on a space device.
In some applications, the quantity of payload data that is collected by and transmitted from a space device to an Earth-bound device approaches or even exceeds the physical limits of the communication link between the space device and the Earth-bound device. One approach to reducing the quantity of payload data that is communicated from a space device to an Earth-bound device is to increase the amount of processing that is performed on the space device. In other words, the space device processes the raw payload data that otherwise would be included in the downstream data. Typically, the resulting processed data is significantly smaller in size than the raw payload data. The resulting data from such processing is then transmitted from the space device to the Earth-bound device as the downstream data.
One way to process raw payload data on a space device employs application-specific integrated circuits (ASICs). Application-specific integrated circuits, while efficient, typically are mission-specific and have limited scalability, upgradeability, and re-configurability. Another way to process raw payload data makes use of anti-fuse field programmable gate arrays (FPGAs). Such an approach typically lowers implementation cost and time. Also, anti-fuse FPGAs typically exhibit a high degree of tolerance to radiation. However, anti-fuse FPGAs are typically not re-programmable (that is, reconfigurable). Consequently, an anti-fuse FPGA that has been configured for one application is not reconfigurable for another application.
Another way to process such raw payload data makes use of re-programmable FPGAs. However, re-programmable FPGAs are typically susceptible to single event upsets. A single event upset (SEU) occurs when an energetic particle penetrates the FPGA (or supporting) device at high speed and high kinetic energy. For example, the energetic particle can be an ion, electron, or proton resulting from solar radiation or background radiation in space. The energetic particle interacts with electrons in the device. Such interaction can cause the state of a transistor in an FPGA to reverse. That is, the energetic particle causes the state of the transistor to change from a logical “0” to a logical “1” or from a logical “1” to a logical “0.” This is also referred to here as a “bit flip.” The interaction of an energetic particle and electrons in an FPGA device can also introduce a transient current into the device.
Payload data applications continue to operate with high amounts of communication interference. Current monitoring techniques limit the re-programmable FPGAs from recovering within a minimal recovery time. The recovery time from one or more single event upsets is critical, especially in operating environments susceptible to high amounts of radiation.
SUMMARYIn one embodiment, a reconfigurable computer is provided. The computer includes a controller and at least one reconfigurable processing element communicatively coupled to the controller. The controller is operable to read at least a first portion of a respective configuration of each of the plurality of reconfigurable processing elements and refresh at least a portion of the respective configuration of the reconfigurable processing element if the first portion of the configuration of the reconfigurable processing element has changed since the first portion was last checked.
DRAWINGSFIG. 1 is a block diagram of an embodiment of a space payload processing system;
FIG. 2 is a block diagram of an embodiment of a reconfigurable computer for use in payload processing on a space device;
FIG. 3 is a block diagram of an embodiment of a configuration interface for a reconfigurable computer; and
FIG. 4 is a flow diagram illustrating an embodiment of a method for controlling at least two reconfigurable processing elements.
DETAILED DESCRIPTIONFIG. 1 is a block diagram of an embodiment of a spacepayload processing system100, as described in the '888 application. Embodiments ofsystem100 are suitable for use, for example, in space devices such as satellites and space vehicles.System100 includes sensor modules1021-1to1022-2. Each of sensor modules1021-1to1022-2is a source of raw payload data that is to be processed bysystem100. It is to be understood, however, that in other embodiments, other sources of raw payload data are used.
Each of sensor modules1021-1to1022-2comprise sensors1031-1to1032-2. In one embodiment, sensors1031-1to1032-2comprise active and/or passive sensors. Each of sensors1031-1to1032-2generate a signal that is indicative of a physical attribute or condition associated with that sensor103. Sensor modules1021-1to1022-2include appropriate support functionality (not shown) that, for example, perform analog-to-digital conversions and drive the input/output interface necessary to supply sensor data to other portions ofsystem100. It is noted that for simplicity in description, a total of four sensor modules1021-1to1022-2and four sensors1031-1to1032-2are shown inFIG. 1. However, it is understood that in other embodiments ofsystem100 different numbers of sensor modules101 and sensors103 (for example, one or more sensor modules and one or more sensors) are used.
For example, in one embodiment, each of sensor modules1021-1to1022-2includes an array of optical sensors such as an array of charge coupled device (CCD) sensors or complimentary metal oxide system (CMOS) sensors. In another embodiment, an array of infrared sensors is used. The array of optical sensors, in such an embodiment, generates pixel image data that is used for subsequent image processing insystem100. In other embodiments, other types of sensors are used.
The data output by sensor modules1021-1to1022-2comprise raw sensor data that is processed bysystem100. More specifically, the sensor data output by1021-1to1022-2is processed byreconfigurable computers1041to104N. For example, in one embodiment where sensor modules1021-1to1022-2output raw image data,reconfigurable computers1041to1042perform one or more image processing operations such as RICE compression, edge detection, or Consultative Committee of Space Data Systems (CCSDS) protocol communications.
The processed sensor data is then provided to back-end processors1061and1062. Back-end processors1061and1062receive the processed sensor data as input for high-level control and communication processing performed byreconfigurable computers1041and1042. In the embodiment shown insystem100, back-end processor1062assembles appropriate downstream packets that are transmitted via acommunication link108 to an Earth-bounddevice110. At least a portion of the downstream packets include the processed sensor data (or data derived from the processed sensor data) that was received fromreconfigurable computers1041and1042. The communication of payload-related data within and between the various components ofsystem100 is also referred to here as occurring in the “data path.” It is noted that for simplicity in description, a total of tworeconfigurable computers1041and1042and two back-end processors1061and1062are shown inFIG. 1. However, it is understood that in other embodiments ofsystem100 different numbers ofreconfigurable computers104 and back-end processors106 (for example, one or more reconfigurable computers and one or more back-end processors) are used.
System100 also includessystem controller112.System controller112 monitors and controls the operation of the various components ofsystem100. For example,system controller112 manages the configuration and reconfiguration ofreconfigurable computers1041and1042.System controller112 is further responsible for control of one or more programmable reconfiguration refresh and readback intervals. Communication of control data within and between the various components ofsystem100 is also referred to here as occurring in the “control path.”
Reconfigurable computers1041and1042are capable of being configured and re-configured. For example,reconfigurable computers1041and1042are capable of being configured and re-configured at runtime. That is, processing that is performed byreconfigurable computers1041and1042is changed while thesystem100 is deployed (for example, while thesystem100 is in space). In one embodiment, each ofreconfigurable computers1041and1042is implemented using one or more reconfigurable processing elements. One such embodiment is described in further detail below with respect toFIG. 2.
In one embodiment, re-configurability ofreconfigurable computers1041and1042is used to fix problems in, or add additional capabilities to, the processing performed by each ofreconfigurable computers1041and1042. For example, whilesystem100 is deployed, new configuration data forreconfigurable computer1041is communicated from earth-bounddevice110 tosystem100 overcommunication link108.Reconfigurable computer1041uses the new configuration data to reconfigure reconfigurable computer1041(that is, itself).
Further, the re-configurability ofreconfigurable computers1041and1042allowsreconfigurable computers1041and1042to operate in one of multiple processing modes on a time-sharing basis. For example, in one usage scenario,reconfigurable computer1042is configured to operate in a first processing mode during a first portion of each day, and to operate in a second processing mode during a second portion of the same day. In this way, multiple processing modes are implemented with the samereconfigurable computer1042to reduce the amount of resources (for example, cost, power, and space) used to implement such multiple processing modes.
Insystem100, each ofreconfigurable computers1041and1042and each of back-end processors1061and1062are implemented on a separate board. Each of the separate boards communicates control information with one another overcontrol bus114 such as a Peripheral Component Interconnect (PCI) bus or a compact PCI (cPCI) bus.Control bus114, for example, is implemented inbackplane116 that interconnects each of the boards. In the example embodiment ofsystem100 shown inFIG. 1, at least some of the boards communicate with one another over one or more data busses118 (for example, one or more buses that support the RAPIDIO® interconnect protocol). In such an implementation, sensor modules1021-1to1022-2are implemented on one or more mezzanine boards. Each mezzanine board is connected to a corresponding reconfigurable computer board using an appropriate input/output interface such as the PCI Mezzanine Card (PMC) interface.
FIG. 2 is a block diagram of an embodiment of areconfigurable computer104 for use in payload processing on aspace device200. The embodiment ofreconfigurable computer104 shown includes reconfigurable processing elements (RPEs)2021and2022, similar to the RPEs described in the '888 application. As similarly noted in the '888 application, embodiments ofreconfigurable computer104 are suitable for use in or withsystem100 as described with respect toFIG. 1 above. It is to be understood that other embodiments and implementations ofreconfigurable computer104 are implemented in other ways (for example, with two or more RPEs202).
RPEs2021and2022comprise reconfigurable FPGAs2041and2042that are programmed by loading appropriate programming logic (also referred to here as an “FPGA configuration” or “configuration”) as discussed in further detail below. Each RPE2021and2022is configured to perform one or more payload processing operations.Reconfigurable computer104 also includes input/output (I/O) interfaces2141and2142. Each of the two I/O interfaces2141and2142are coupled to a respective sensor module102 ofFIG. 1 that receives raw payload data for processing by the reconfigurable processing elements202.
I/O interfaces2141and2142and RPEs2021and2022are coupled to one another with a series of dual-port memory devices2161to2166. This obviates the need to use multi-drop buses (or other interconnect structures) that are more susceptible to one or more SEUs. Each of a first group of dual-port memory devices2161to2163has a first port coupled to I/O interface2141. I/O interface2141uses the first port of each of memory devices2161to2163to read data from and write data to each of memory devices2161to2163. RPE2021is coupled to a second port of each of memory devices2161to2163. RPE2021uses the second port of each of memory devices2161to2163to read data from and write data to each of memory devices2161to2163. Each of a second group of three dual-port memory devices2164to2166has a first port coupled to I/O interface2142. I/O interface2142uses the first port of each of memory devices2164to2166to read data from and write data to each of memory devices2164to2166. RPE2022is coupled to a second port of each of memory devices2164to2166. RPE2022uses the second port of each of memory devices2164to2166to read data from and write data to each of memory devices2164to2166.
In this example embodiment, I/O interfaces2143and2144are RAPIDIO interfaces. Each of RAPIDIO interfaces2143and2144are coupled to a respective back-end processor106 ofFIG. 1 over one ormore data buses118 inbackplane116 that supports the RAPIDIO interconnect protocol. Each of RPEs2022to2022is coupled to a respective one of the RAPIDIO interfaces2143and2144in order to communicate with the one or more back-end processors106 ofFIG. 1.
Reconfigurable computer104 further includessystem control interface208.System control interface208 is coupled to each of RPEs2022to2022overconfiguration bus218.System control interface208 is also coupled to each of I/O interfaces2141and2142oversystem bus220.System control interface208 provides an interface by which thesystem controller112 ofFIG. 1 communicates with (that is, monitors and controls) RPEs2022to2022and one or more I/O devices coupled to I/O interfaces2141and2142.System control interface208 includescontrol bus interface210.Control bus interface210 couplessystem control interface208 to controlbus114 ofFIG. 1.System control interface208 andsystem controller112 communicate overcontrol bus114. In one implementation,control bus interface210 comprises a cPCI interface.
System control interface208 also includeslocal controller212.Local controller212 carries out various control operations under the direction ofsystem controller112 ofFIG. 1.Local controller212 performs various FPGA configuration management operations as described in further detail below with respect toFIG. 3. The configuration management operations performed bylocal controller212 include reading an FPGA configuration fromconfiguration memory206 and loading the FPGA configuration into each of reconfigurable FPGAs2041and2042. One or more FPGA configurations are stored inconfiguration memory206. In one implementation,configuration memory206 is implemented using one of a flash random access memory (Flash RAM) and a static random access memory (SRAM). In other embodiments, the one or more FPGA configurations are stored in a different location (for example, in a memory device included in system controller112). The configuration management operations performed bylocal controller212 also include SEU mitigation. Examples of SEU mitigation include periodic and/or event-triggered refreshing of the FPGA configuration and/or FPGA configuration readback and compare. In one embodiment, the SEU mitigation described here (and with respect toFIG. 4 below) is performed bylocal controller212 for each ofRPEs2021and2022 that sustain at least one substantial SEU.
In the example embodiment shown inFIG. 2,system control interface208 andconfiguration memory206 are implemented using radiation-hardened components and reconfigurable processing elements2021and2022(including reconfigurable FPGAs2041and2042), I/O interfaces2141to2144, and dual-port memory devices2161to2166are implemented using commercial off the shelf (COTS) components that are not necessarily radiation hardened. COTS components are less expensive, more flexible, and easier to program. Typically, the processing performed in the data path changes significantly more than the processing performed in the control path from mission-to-mission or application-to-application. Using COTS components allowsreconfigurable computer104 to be implemented more efficiently (in terms of time, cost, power, and/or space) than radiation-hardened components such as non-reconfigurable, anti-fuse FPGAs or ASICs. Moreover, by incorporating SEU mitigation techniques insystem control interface208, redundancy-based SEU mitigation techniques such as triple modular redundancy are unnecessary. This reduces the amount of resources (for example, time, cost, power, and/or space) needed to implementreconfigurable computer104 for use in a given space application with COTS components.
FIG. 3 is a block diagram of an embodiment of aconfiguration interface300, for a reconfigurable computer.Configuration interface300 compriseslocal controller212,configuration memory206,control bus interface210, andconfiguration bus218.Configuration memory206,control bus interface210, andconfiguration bus218 were described above with respect toFIG. 2.Local controller212 comprisesinternal bus controller302, RPE CRC generators3061and3062, and RPE interface controllers3081and3082. It is to be understood that other embodiments and implementations oflocal controller212 are implemented in other ways (for example, with two or more RPE CRC generators306 and two or more RPE interface controllers308). Internal bus controller further includesinternal arbiter304.Internal bus controller302 is directly coupled to each of RPE CRC generators3061and3062by inter-core interfaces3201and3202, respectively. Inter-core interfaces3201and3202are internal bi-directional communication interfaces.Internal arbiter304 is directly coupled to each of RPE interface controllers3081and3082by arbiter interfaces3221and3222, respectively. Arbiter interfaces3221and3222are internal bi-directional communication interfaces.Internal arbiter304 prevents inter-core communications withinlocal controller212 from occurring concurrently, which may result in an incorrect operation. Each of RPE CRC generators3061and3062is directly coupled to RPE interface controllers3081and3082by CRC interfaces3241and3242, respectively. CRC interfaces3241and3242are internal bi-directional communication interfaces.
Internal bus controller302 is coupled to configuration memory206 (shown inFIG. 2) byconfiguration memory interface316.Configuration memory interface316 is an inter-component bi-directional communication interface.Internal bus controller302 is also coupled control bus interface210 (shown inFIG. 2) bycontroller logic interface318.Controller logic interface318 is an inter-component bi-directional communication interface. In one implementation,controller logic interface318 is one of a WISHBONE interface, a cPCI interface, or the like. Each of RPE interface controllers3081and3082are coupled toconfiguration bus218 for communication with RPE2021and2022ofFIG. 2. Each of RPE interface controllers3081and3082further include readback controllers3101and3102, arbiters3121and3122, and configuration controllers3141and3142, respectively, whose operation is further described below. In one implementation,internal arbiter304 and each of arbiters3121and3122are two-interface, rotational-arbitration state machines. Other implementations are possible.
In operation, a full or partial set of configuration data for each of RPEs2021to2022is retrieved fromconfiguration memory206 byinternal bus controller302. In this example embodiment, system controller112 (ofFIG. 1) determines whether a full or partial set of configuration data is to be analyzed.System control interface208 is capable of operating at a 50 MHz clock rate (maximum) and will complete one data transfer (for example, a data frame or byte) on every rising edge of the clock during a burst read (readback) or burst write (configuration) operation. In one implementation,local controller212 operates at a clock rate of 10 MHz.Internal arbiter304 determines the order in which each RPE interface controllers3081and3082receive the configuration data without causing an interruption in operation ofreconfigurable computer104.
Once each of RPE interface controllers3081and3082receive the configuration data, each of readback controllers3101and3102controls a readback operation of the configuration data. For every readback operation of the configuration data, each of RPE CRC generators3061and3062perform a CRC on a full or partial set of the configuration data. The CRC determines if any configuration data bits have changed since a previous readback of the same configuration data (that is, corrupted due to one or more SEUs). In a situation where a readback CRC calculation does not match a stored CRC,local controller212 enters an auto-reconfiguration mode. In the example embodiment ofFIG. 3, auto reconfiguration due to a CRC error is a highest priority. Additionally,local controller212 provides a CRC error count register for gathering of SEU statistics.
Local controller212 supports interleaving of readback and reconfiguration (refresh) operations by interleaving priority and order via arbiters3121and3122. Arbiters3121and3122are each responsible for arbitration of the configuration data between RPE CRC generator3061(3062) and configuration controller3141(3142). Each of configuration controllers3141and3142take in one or more input requests from an internal register file (not shown) and decode which operation to execute. Configuration controller3141(3142) identifies a desired operation to be executed and makes a request for the transaction to be performed by supporting logic withinlocal controller212.
Each of configuration controllers3141and3142select an operating mode for multiplexing appropriate data and control signals internally. Once all requested inputs are received, configuration controller3141(3142) decides which specific request to execute. Once the specific request is granted, configuration controller3141(3142) issues an access request to arbiter3121(3122) for access to complete the request. Each request is priority-encoded and implemented in a fair arbitration scheme so no single interface is rejected of a request to accessconfiguration bus218. Each of configuration controllers3141and3142provide a set of software instructions forlocal controller212 with the capability to interface toconfiguration bus218 on a cycle-by-cycle basis. Specifically, upon receipt of the access request, configuration controller3141(3142) outputs the configuration data fromconfiguration memory206 onconfiguration bus218.
Local controller212 andcontrol bus interface210 provide one or more independent configuration buses (for example, RPE interface controllers3081and3082). In one implementation, RPE interface controllers3081and3082provide simultaneous readback and CRC checking for each of RPE2021and2022. Subsequently, simultaneous readback of one configuration of RPE2021(2022) will occur while RPE2022(2021) is reconfigured. Further,local controller212 provides one or more programmable reconfiguration refresh and readback interval rates.Local controller212 also supports burst read and burst write access. In one implementation, wait states are inserted during back-to-back read/write and write/read operations. Full and partial reconfiguration of RPEs2021and2022occurs within a minimum number of operating cycles and substantially faster than previous (that is, software-based) SEU mitigation operations.
FIG. 4 is a flow diagram illustrating amethod400 for controlling at least two reconfigurable processing elements. In the example embodiment shown inFIG. 4,method400 is implemented usingsystem100 andreconfigurable computer104 ofFIGS. 1 and 2, respectively. In particular, at least a portion ofmethod400 is implemented bylocal controller212 ofsystem control interface210. In other embodiments, however,method400 is implemented in other ways.
Once a refresh interval value is established (or adjusted) atblock404,method400 begins the process of monitoring the configuration of each available RPE for possible corruption due to an occurrence of a single event upset. A primary function ofmethod400 is to automatically reconfigure a corrupted configuration of a RPE within a minimum amount of operating cycles. In one implementation,method400 substantially improves completion time for a full or partial refresh or reconfiguration to maintain operability of the space payload processing application.
A determination is made about whether the refresh interval rate has changed from a previous or default level (checked in block406). This determination is made insystem controller112 described above with respect toFIG. 1. If the refresh interval level has changed, thesystem controller112 transfers the refresh interval level to thelocal controller212 atblock408, and proceeds to block410. If the refresh interval level has not changed, or the refresh interval level is fixed at a (static) predetermined level,method400 continues atblock410. Atblock410, a determination is made about whether the current refresh interval has elapsed. Until the refresh interval elapses, processing associated withblock410 is repeated.
Atblock412,method400 begins evaluating the configuration status for a RPE (referred to here as the “current” RPE) by performing a readback operation. In one implementation of such an embodiment, the readback operation is performed by the RPE interface controller308 for the current RPE. Thelocal controller212 reads the current configuration of the reconfigurable FPGA for the current RPE and compares at least a portion of the read configuration to a known-good value associated with the current configuration. If the read value does not match the known-good value, the configuration of the current RPE is considered corrupt. In one implementation, such a readback operation is performed by reading each byte (or other unit of data) of the configuration of the FPGA for the current RPE and comparing that byte to a corresponding byte of the corresponding configuration stored inconfiguration memory206. In other words,local controller212 performs a byte-by-byte compare. In another implementation, one or more CRCs (or other error correction code) values are calculated for the current configuration of the FPGA for the current RPE by a respective RPE CRC generator.
If the configuration for the current RPE is corrupt (checked in block414),method400 begins a full or partial reconfiguration (refresh) of the current RPE202 atblock416. The determination as to whether to perform a full or partial reconfiguration is made bysystem controller112 ofFIG. 1. If the readback operation performed inblock412 does not reveal corruption of the configuration of the current RPE202,method400 proceeds directly to block418.
Atblock418,method400 determines whether all available RPEs have been evaluated. If not,method400 returns to block412 to evaluate the configuration status for the next available RPE. When all available RPEs have been evaluated,method400 waits until at least one of the available RPEs is substantially functional (checked in block422) at whichtime method400 returns to block404.
In one example of the operation ofmethod400 in thesystem100 ofFIG. 1, when RPE2021is to be configured (or reconfigured), an appropriate configuration is read fromconfiguration memory206 and loaded into thereconfigurable FPGA2041. Similar operations occur to configure RPE2022. Each of RPE2021and RPE2022is configured, for example, when thereconfigurable computer104 ofFIG. 2 initially boots after an initial system power on or after a system reset. In some embodiments ofreconfigurable computer104 that support timesharing multiple operating modes, thereconfigurable computer104 is configured so that each time the operating mode ofreconfigurable computer104 changes, the configuration for the new operating mode is read fromconfiguration memory206 and loaded into the reconfigurable FPGA for the respective RPEs. Also, in such an example, RPE2021andRPE2022 are configured to perform, as a part of one or more SEU mitigation operations, a “refresh” operation in which the configuration of the respective reconfigurable FPGA2041and FPGA2042is reloaded.