US20060230403A1 - Coordination in a distributed system - Google Patents

Abstract

Respective coordinators are spawned or activated to coordinate activities with regard to respective tasks. Where the respective tasks require cooperative efforts of a plurality of controllers, the respective coordinators ensure cooperative efforts by generating and communicating cooperative commands to the plurality of controllers. The coordinators may act as clearinghouses for system data, selectively requesting and relaying system information to appropriate controllers. For example, a document processing system activates respective coordinators for respective sheets of print media. The respective coordinators orchestrate the transportation of the sheets by sequentially orchestrating the activities of sequentially selected pluralities of transportation actuator controllers. Selected sheet position information from sensors and/or from models maintained by the actuator controllers may be relayed by the coordinators to selected actuator controllers as appropriate to the sheet transportation tasks.

Description

CROSS REFERENCE
• The following applications, the disclosures of each being totally incorporated herein by reference are mentioned: U.S. patent application Ser. No. ______, filed, for Communication in a Distributed System by Markus P. J. Fromherz, et al. (20041213-US-NP, XERZ 2 00864); U.S. patent application Ser. No. ______, filed, for Synchronization in a Distributed System by Lara S. Crawford, et al. (20041209-US-NP, XERZ 2 00862); and U.S. patent application Ser. No. ______, filed, for On-The-Fly State Synchronization in a Distributed System by Haitham A. Hindi, et al. (20041214-US-NP, XERZ 2 00865).
BACKGROUND
• There is illustrated herein in embodiments, an architecture including methods and systems for coordinating activities in a distributed system. For example, a distributed system may include a collection of modules, each with its own function. The collection of modules is interconnected to carry out a particular function or functions. The interconnection may be physical and/or logical in nature. Modules may be connected by a network or other communications scheme. Communications media may include wire, coaxial cable, fiber optics and/or radio frequency (RF) transmissions. Embodiments will be described with reference to document processing systems. However, embodiments of the architecture may be beneficially applied in a wide variety of control system environments.
• Document processors include, for example, printers, copiers, facsimile machines, finishers and devices for creating documents, such as word processors and desk top publishers. In some instances, document processors provide the services of two or more of these devices. For instance, document processors that provide printing, copying, scanning, and faxing services are available. Printers and copiers can include finishers that staple, shrink wrap or otherwise bind system output. Finishers may also fold or collate documents.
• In order to increase throughput, some printers and copiers are being developed which include two or more marking engines. For example, U.S. patent application Ser. No. 10/924,113 filed Aug. 23, 2004 by Jonas M. M. deJong, et al. for a Printing System with Inverter Disposed for Media Velocity Buffering and Registration; U.S. patent application Ser. No. 10/924,106 filed Aug. 23, 2004 by Robert M. Lofthus, et al. for a Printing System with Horizontal Highway and Single Pass Duplex; U.S. patent application Ser. No. 10/924,459 filed Aug. 23, 2004 by Barry P. Mandel, et al. for a Parallel Printing Architecture Consisting of Containerized Image Marking Engine Modules; U.S. patent application Ser. No. 10/860,195 filed Jun. 6, 2004 by Robert M. Lofthus, et al. for a Universal Flexible Plural Printer to Plural Finisher Sheet Integration System; U.S. patent application Ser. No. 10/881,619 filed Jun. 30, 2004 by Daniel G. Bobrow for a Flexible Paper Path Using Multidirectional Path Modules; U.S. patent application Ser. No. 10/761,522 filed Jan. 21, 2004 by Barry P. Mandel, et al. for a High Print Rate Merging and Finishing System for Parallel Printing; U.S. patent application Ser. No. 10/785,211 filed Feb. 24, 2004 by Robert M. Lofthus, et al. for a Universal Flexible Plural Printer to Plural Finisher Sheet Integration System; and U.S. patent application Ser. No. 10/917,768 filed Aug. 13, 2004 by Robert M. Lofthus for a Parallel Printing Architecture Consisting of Containerized Image Marking Engines and Media Feeder Modules, all of which are incorporated herein by reference, describe aspects of tightly integrated document processing systems including a plurality of marking engines.
• Additionally, some printers and copiers are being developed using a hypermodular structure to increase modularity and flexibility. These systems may possess a number of distributed processors, sensors, and actuators. For example, U.S. patent application Ser. No. 10/357,687 filed Feb. 4, 2003 by David K. Biegelsen, et al., for Media Path Modules; U.S. patent application Ser. No. 10/357,761 filed Feb. 4, 2003 by Markus P. J. Fromherz, et al., for Frameless Media Path Modules; U.S. patent application Ser. No. 10/740,705 filed Dec. 19, 2003 by David K. Biegelsen, et al., for a Flexible Director Paper Path Module; and U.S. patent application Ser. No. 10/812,376 filed Mar. 29, 2004 by David G. Duff, et al., for a Rotational Jam Clearance Apparatus, all of which are incorporated herein by reference, describe aspects of tightly integrated document processing systems including hypermodules.
• Some systems, including some document processing systems, are based on a centralized control architecture wherein a single computational platform controls all system actuators and receives all system feedback information. These architectures work well where the systems are relatively small and are of a fixed or unchanging configuration. However, as system size increases, the computational capabilities of a single platform can be overwhelmed. Additionally, providing individual interfaces between the single computational platform and each of the sensors and actuators of the system can be impractical. Furthermore, where it is desirable to assemble or reconfigure a system from various subcomponents, the direct interfacing of sensors and actuators to the central platform becomes problematic.
• These factors have led to the development of systems based on network communications. For example, U.S. Pat. No. 6,615,091 B1 to Birchenough, et al. for a Control System and Method Therefore allegedly disclosed an embodiment of a distributed control system including a main control coordinator, three local process station controllers and a designated number of process module controllers, each associated with a process module. The control system allegedly provides a real time operating system and has a communication bus platform provided via an Ethernet™ communication bus and a second bus to connect the controllers in a distributed control network. The Ethernet™ bus connects the main control coordinator and each of the local process station controllers and a continuous motion conveyer controller. Each of the process module controllers are connected via the second bus to designated local process station controllers.
• In the system of Birchenough, the main controller agent interacts with each of the process station agents, and each of the process station agents interacts with each of the process module agents that are assigned thereto. During normal manufacturing operation, the main controller coordinator agent sends article notice messages to the process station agents to notify the process station agents of the oncoming articles of manufacture. A process station normally will not process the article of manufacture unless the process station agent that controls a particular process module has received an article notice message indicating that it should do so and the continuous feed indexer has returned a report that it is in proper position. In response, the process station agent notifies the designated process module agent to initiate its programmed process operation. Once the process module has completed its intended operation, the process module agent issues a work report message which is sent to the process station agent. The process station agent then broadcasts the work report message to other process stations as well as to the main control coordinator.
• It appears that in the system of Birchenough, et al., a single entity (e.g., the main coordinator) is aware of and maintains information regarding each task, object or workpiece being processed by the system. This may limit the scalability of the system. For example, as the size of the system increases, the capabilities and/or resources of the main control coordinator (or processor running the main control coordinator) may be overwhelmed.
• Accordingly, there has been a desire for a distributed control architecture, including systems and methods, that provides for scalability and reconfigurability in a modular system environment.
BRIEF DESCRIPTION
• A method for coordinating controllers in a distributed control system that is operative to perform a plurality of simultaneous tasks includes determining a respective task of the plurality to be performed and activating a respective coordinator in association with the respective task. The respective coordinator may then identify a plurality of respective subtasks to be performed in order to complete the respective task, identify a plurality of respective controllers for controlling a plurality of actuators to perform the respective subtasks, generate respective commands for directing the performance of the respective plurality of subtasks and communicate the respective commands to the respective controllers as appropriate to the respective subtasks. The respective coordinator may also identify at least one respective information source that is able to provide progress information regarding the performance of the respective subtasks, collect the respective progress information from the respective at least one information source and communicate the respective progress information to the respective controllers as appropriate to the respective subtasks.
• The method may also include deactivating respective controllers as respective subtasks are determined to be completed based on the collected respective progress information, thereby releasing the respective controllers to execute commands communicated from another coordinator associated with another task and/or deactivating the coordinator when the respective task is determined to be completed. For example, one or more model may be maintained to estimate progress of the respective subtasks and the coordinator and/or controllers may be deactivated when the respective subtasks are completed according to the one or more models.
• For instance the models can be based on commands the coordinator generates and/or on progress information received from progress information sources. The output of command based models and progress information models can be compared to determine a task progress error value. The error value can be compared to an error limit. An error task, such as generating an error message or taking corrective or compensatory action, can be performed if the error value is beyond the error limit.
• Some embodiments include a method for coordinating controllers in a distributed control system that is operative to simultaneously perform operations on a plurality of workpieces. These embodiments can include activating a respective coordinator for each respective workpiece, wherein each respective coordinator encompasses knowledge regarding an itinerary of operations to be applied to each respective workpiece. Each respective coordinator may then issue respective commands to a series of actuator controllers for directing a series of actuators to perform respective operations on the respective workpiece according to the itinerary, maintain at least one respective estimate of progress of the respective operations performed on the respective workpiece and deactivate the respective coordinator when the itinerary of operations is completed.
• A method for processing a sheet in a document processing system can include receiving a sheet description specifying a document processing job to be performed on the sheet and activating a coordinator as a proxy for the sheet. The coordinator may then identify a plurality of respective subtasks to be performed in order to process the sheet according to the sheet description, identify, based on the identified respective subtasks, a plurality of respective controllers for controlling a plurality of actuators to perform the respective subtasks, generate respective commands for directing the respective plurality of subtasks and communicate the respective commands to the respective controllers as appropriate to the respective subtasks. The coordinator may also identify at least one respective information source that is able to provide progress information regarding the performance of the respective subtasks, collect the respective progress information from the respective at least one information source, communicate the respective progress information to the respective controllers as appropriate to the respective subtasks, estimate the respective progress of the respective subtasks and deactivate the respective controllers as the subtasks are completed. The coordinator may be deactivated when the task is completed.
• A system that is operative to perform embodiments of the methods can include a plurality of controllers, at least one supervisor element, and a plurality of information sources. The at least one supervisory element may be operative (alone or in combination) to generate respective task descriptions describing respective tasks, to activate respective coordinators to be responsible for orchestrating completion of the respective tasks and to communicate the respective task descriptions to the respective coordinators. The plurality of controllers may be operative to control a plurality of actuators according to respective subtask descriptions received from the respective coordinators. The plurality of information sources may be operative to report status information regarding respective progress of respective tasks to the respective coordinators. For example, the information sources may be sensors of the system or models or estimators of the controllers. The respective coordinators may be operative to receive the respective task descriptions, identify, based on the respective task descriptions, a plurality of respective subtasks to be performed in order to complete the task, identify, based on the respective subtasks, a subset of the plurality of respective controllers, for controlling a subset of the plurality of actuators to perform the respective subtasks, identify respective subsets of the plurality of information sources that are able to provide progress information regarding the performance of the respective subtasks, generate respective commands for performing the respective plurality of subtasks, communicate the respective commands to the respective controllers as appropriate to the respective subtasks, collect the respective progress information from the respective subsets of information sources and communicate the respective progress information to the respective controllers as appropriate to the respective subtasks.
• A document processing system embodiment can include one or more marking engines, including, for example a xerographic marking engine. Additionally, the system can include a transportation system that is operative to transport sheets of print media to and/or from the first marking engine. Some embodiments may also include a plurality of information sources that are operative to report status information regarding respective progress of respective sheet processing tasks to the respective sheet coordinators. The transportation system can include a plurality of transport actuators, at least one supervisory element, a plurality of respective transport controllers and a module controller. The at least one supervisory element can be operative (alone or in combination) to generate respective sheet processing task descriptions describing respective sheet processing tasks and to activate respective sheet coordinators to be responsible for orchestrating the respective sheet processing tasks according to the respective sheet processing task descriptions. The plurality of respective transport controllers can be operative to control respective sets of transport actuators, of the plurality of transport actuators, according to respective sheet processing subtask descriptions received from the respective sheet coordinators. The first marking module controller may be operative to control aspects of processing of a first marking engine according to respective sheet processing subtask descriptions received from the respective sheet coordinators.
• For example, the respective sheet coordinators are operative to receive the respective sheet processing task descriptions, identify, based on the respective sheet processing task descriptions, a plurality of respective sheet processing subtasks to be performed in order to complete the respective sheet processing tasks, identify, based on the respective sheet processing subtasks, a subset of the plurality of respective transport controllers and the first marking module controller, for controlling a subset of the plurality of transport actuators and the first marking engine to perform the respective sheet processing subtasks, identify respective subsets of the plurality of information sources that are able to provide progress information regarding the performance of the respective sheet processing subtasks, generate respective commands for performing the respective plurality of sheet processing subtasks, communicate the respective commands to the respective transport controllers and/or first marking engine controller as appropriate to the respective subtasks, collect the respective progress information from the respective subsets of information sources and communicate the respective progress information to the respective transportation controllers and/or first marking engine controller as appropriate to the respective sheet processing subtasks.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a block diagram of a system wherein a supervisory element has activated, spawned or instantiated a plurality of respective coordinators that are responsible for orchestrating respective tasks.
• FIG. 2 is a flow chart outlining a method for coordinating controllers.
• FIG. 3 is a flow chart outlining a method for coordinating controllers that includes collection and relaying of information.
• FIG. 4 is a block diagram of a document processing system wherein a supervisory element has activated, spawned or instantiated a plurality of respective coordinators that are responsible for orchestrating respective document processing tasks.
DETAILED DESCRIPTION
• Referring toFIG. 1, afirst system104 embodiment includes aplurality106 of controllers. For example, theplurality106 of controllers includes a first, second, third, fourth andfifth controller108,110,112,114,116. The controllers may, for example, be associated with actuators and sensors. For instance, the first, second andthird controllers108,110,112 are associated with first, second and third sets ofactuators118,120,122 and first, second and third sets ofsensors124,126,128. Thefourth controller114 is associated with a fourth set ofactuators130. Thefifth controller116 is associated with a fourth set ofsensors132. Theactuators118,120,122,130 andsensors124,126,128,132 manipulate or sense objects in, or aspects of, respective portions of thesystem104. For example, the first set ofactuators118 and first set ofsensors124 are associated with afirst portion140 of thesystem104. The second set ofactuators120 and the second set ofsensors126 are associated with asecond portion142 of thesystem104. The third set ofactuators122 and the third set ofsensors128 are associated with athird portion144 of thesystem104. The fourth set ofactuators130 are associated with afourth portion146 of thesystem104 and the fourth set ofsensors132 are associated with afifth portion148 of thesystem104.
• Some or all of the system portions may be tightly coupled. Tightly coupled systems or system portions are those wherein the performance or activities of a first system portion has an effect on the performance or activities of a second portion, even to the point where, if the activities of the first portion and the second portion are not coordinated, they may interfere with or disrupt each other. For instance, in an automotive system, an engine/transmission subsystem may be considered to be tightly coupled with a braking subsystem because an uncoordinated application of the braking system may interfere with or prevent the engine/transmission system from propelling a vehicle.
• In the embodiment illustrated inFIG. 1, first, second and third elements ofsystem dynamics152,154,156 tightly couple thesecond system portion142 to thethird system portion144, tightly couple thethird system portion144 to thefourth system portion146 and tightly couple thefourth system portion146 to thefifth system portion148. Thefirst system portion140 is illustrated as having only a loose or minimal interaction with thesecond system portion142 and is not tightly coupled thereto.
• Thefirst system104 may also include asupervisor160 or supervisory element. For example, thesupervisory element160 may be a scheduler and/or a planner. Thesupervisor160 determines which tasks are to be performed or which workpieces are to be processed. This element and/or one or more other supervisory elements activate, spawn or instantiate a separate coordinator for each task or workpiece. For example, afirst coordinator170 is activated or spawned in association with a first task or workpiece, and asecond coordinator180 is activated or spawned in association with a second task or workpiece. Thecoordinators170,180 are activated and initialized in such a manner as to prevent interference between the coordinators.
• For example, if the first task or workpiece and the second task or workpiece both require the services of the first, second, third, fourth andfifth system portions140,142,144,146,148, then, for example, thefirst coordinator170 is activated and takes control of thefirst system portion140 by communicating with thefirst controller108. The activation of thesecond coordinator180 may be delayed until thefirst coordinator170 no longer requires the services of thefirst system portion140. Alternatively, thesecond coordinator180 is activated early and directed to wait or idle until such a time as thefirst coordinator170 no longer needs the services of a first system resource (e.g.,140).
• Thefirst coordinator170 releases thefirst controller108 when the first task or workpiece no longer needs the services of thefirst system portion140. Thefirst coordinator170 may then send commands requesting the services of another system resource (e.g., the second system portion142) for accomplishing a second subtask. Alternatively, thefirst coordinator170 may begin requesting services from the second resource before the first resource has completed a first subtask. In either case, thefirst coordinator170 sequentially sends commands to the controllers (e.g.,110,112,114,116) requesting services of their respective system portions (e.g.,142,144,146,148). When appropriate, thefirst coordinator170 sends coordinated commands to a plurality of controllers. For example, if a subtask requires coordinated activity between two or more system portions at once, then the coordinator generates coordinated commands to two or more controllers associated therewith.
• InFIG. 1, thefirst system104 embodiment is depicted at a point in time wherein the first task or workpiece requires the services of thefourth system portion146 and the first coordinator is communicating with thefourth controller114. Proximate to issuing commands to, or taking control of, thefourth controller114, thethird controller112 may have been deactivated or released from the control of thefirst coordinator170. For example, commands previously issued to thethird controller112 might have been associated with an expiration parameter. The expiration parameter may have been, for example, a time limit or a processing milestone. When an event occurs that matches or surpasses the value of the expiration parameter, thethird controller112 may be deactivated or released from the control of thefirst coordinator170.
• Alternatively, the first workpiece or task may require simultaneous services of both thethird system portion144 and thefourth system portion146. In that case, the first coordinator generates and communicates coordinated or cooperative commands to the third112 and fourth114 controllers.
• At an appropriate point, the first coordinator will generate commands and transmit or communicate them to thefifth controller116 requesting services of thefifth system portion148. If the services of the fifth system portion are required contemporaneously with the services of fourth146 and/or third144 system portions, then thefirst coordinator170 generates and communicates cooperative commands to the fifth116, fourth114 and/or third112 controllers.
• FIG. 1 also illustrates thesecond coordinator180 to be in communication with thesecond controller110. For example, thesecond coordinator180 is requesting services of thesecond system portion142. Thefirst controller108 is being, or has been, released from serving thesecond coordinator180, and thesecond coordinator180 is preparing or will prepare to take control, or request the services of, thethird system portion144 through thethird controller112. Since the second142 and third system portions are tightly coupled152, the second controller may generate and communicate cooperative commands to the second110 and third112 controllers, thereby directing them to perform cooperative operations or processes on the second task or workpiece. In some instances, thefirst coordinator170 and thesecond coordinator180 may communicate with each other to facilitate coordination or for other reasons.
• When thefirst controller108 is released or deactivated, it becomes available to execute commands of yet another coordinator (not shown) which thesupervisor160 may activate, spawn or instantiate to coordinate and orchestrate a third task or workpiece for processing.
• Referring toFIG. 2, afirst method210 for coordinating controllers in a distributed control system can include determining214 a task to be performed by the system and activating218 a coordinator (e.g.,170,180) in association with the task. The coordinator (e.g.,170,180) may then identify222 subtasks to be performed in order to complete the task, identify226 controllers for performing the subtasks, generate230 commands to direct the controllers to perform the subtasks and communicate234 the commands to the controllers as appropriate to the subtasks. The coordinator (e.g.,170,180) may also monitor238 the progress of the subtasks.
• Determining214 may be done by any known task determination method. For example, a supervisory element (e.g.,160) may autonomously determine that a task should be performed. For instance, sensor readings may indicate that a particular action is required. Alternatively, the supervisory element may receive instructions from another device or higher level supervisory device. For example, a system user may enter commands, through a user interface, directing that a task or series of tasks be performed. Alternatively, commands may be received over a network or via a wireless link from another system or remote user. The supervisory element may translate or transform such commands into a task description or specification and activate218 a coordinator in accord with the task specification.
• Activating218 a coordinator in association with the task may include initializing a coordinator according to the task specification. For instance, a coordinator (e.g.,170,180) may be embodied in software run by a computational platform such as a microprocessor. One or more coordinators may be loaded into program memory in an idle state. The supervisory element selects an idle coordinator and initializes it in accord with the task specification, thereby activating the coordinator. Alternatively, activating218 a coordinator may include spawning or instantiating a coordinator by creating or loading a new process or by referencing a program or subprogram. The spawned or instantiated element may be initialized or loaded with the task specification or may simply be selected for spawning or instantiation based on the task specification. In a further alternative, the task specification may be transmitted or otherwise communicated to the coordinator at some point after the coordinator has been activated218.
• Identifying222 subtasks to be performed to complete the task and identifying226 controllers to perform the subtasks are related to each other and may occur simultaneously or contemporaneously. For example, identifying subtasks may be dependent on the capabilities of the controllers (e.g.,108,110,112,114,116) and modules or system portions (e.g.,140,142,144,146,148) associated with the controllers. For instance, if the task specification indicates that a hole is to be drilled in a workpiece, the coordinator (e.g.,170,180) may have to determine how one system portion (e.g.,140,142,144,146,148) manipulating the piece will have to adjust its control to compensate for the forces applied by another system portion doing the drilling.
• In some embodiments, it is assumed that all task specifications are valid. For example, a task specification requiring that a hole be placed in a workpiece would only be communicated to a coordinator if the system included a system portion capable of providing the hole. In other embodiments, the coordinator (e.g.,170,180) checks the validity or feasibility of a task specification and communicates an error or exception message to the requesting entity (e.g.,160) if the specified task is not feasible or valid. In still other embodiments, validity or feasibility is determined by the controllers (e.g.,108,110,112,114,116) and communicated to the coordinators (e.g.,170,180), and from the coordinators to the supervisor (e.g.,160).
• Identifying226 the controllers or system elements to be used to complete the task helps identify222 the subtasks to be performed. For instance, subtasks for the hole-stamping example task mentioned above might include operating a motor in a transport system so as to move the workpiece to a hole-stamping module, operating a set of registration motors to align the workpiece in the hole-stamping module according to a desired hole location defined in the task specification, installing a stamp appropriate for stamping a hole of a size defined in the task specification in a chuck, operating actuators so as to stamp the hole in the workpiece, operating clamps and alignment motors to hold the workpiece in place in while the hole is being stamped, and operating motors in a transport system to move the workpiece out of the stamping module. Several of these subtasks would have to be coordinated; for example, the registration operation must be coordinated with the clamping and stamping operations.
• System configuration and/or capabilities information is made available to the coordinator (e.g.,170,180). For example, the coordinator may be initialized, spawned or instantiated with all the required system configuration information. Alternatively, the coordinator may have access to a database or may be able to query or poll all the available controllers in a system to determine their capabilities and, if appropriate, their relative locations.
• Generating230 commands to direct the performance of the subtasks is also system or embodiment dependent. In the hole stamping example from above, the activated218 coordinator may generate230 a command to direct a first controller (e.g.,108) to drive a workpiece ejection mechanism to eject one workpiece onto a transport mechanism. Another generated command may be for a second controller (e.g.,110), directing the second controller to drive the transport mechanism at a particular speed for a particular period of time or until a sensor indicates the workpiece has arrived at a particular position. Additional commands would be generated230 to load and orient the workpiece in the hole punch module and to select, install and drive the punch as well as transport the finished workpiece to an output device.
• Communicating234 the commands to the controllers as appropriate to the subtasks can include any desired communication mechanism. For example, the activated218 coordinator (e.g.,170,180) may communicate with the controllers (e.g.,108,110,112,114,116) over a system network, such as Ethernet™ or the Controller Area Network (CAN) developed by Robert Bosch GmbH. Information regarding the format or protocol used to communicate with each controller (e.g.,108,110,112,114,116) is made available to the activated218 coordinator, either during initialization or from some system resource such as a memory or database.
• As indicated above, the activated218 coordinator (e.g.,170,180) may optionally monitor the progress of subtasks. For example, the controller may receive subtask completion messages from the controllers as each subtask is completed. Additionally, or alternatively, the activated218 coordinator may establish and maintain a model of each subtask (or the task as a whole). For instance, where a communicated234 command directs a controller to drive an actuator to move a workpiece at a given speed, the activated218 coordinator may maintain a command model that estimates a position of the workpiece based on the commanded speed. The model assumes that the command will be followed. Additionally, or alternatively, the activated218 coordinator may receive progress information from one or more progress information sources. For instance, the activated218 coordinator (e.g.,170,180) may receive progress information from a controller (e.g.,108,110,112,114,116) and/or from sensors (e.g.,124,126,128,132). The activated218 coordinator may maintain a progress model of the subtasks (or task as a whole) based on this progress information or may use the progress information to fine tune or update a command-based model.
• Additionally, or alternatively, progress information and/or progress model information may be compared to output from the command model. If the progress information or progress model information is very different from progress estimates of the command model (i.e., the difference is beyond an error threshold), the activated218 coordinator (e.g.,170,180) may perform some error or exception handling task. For instance, a coordinator may send an error message or warning to a supervisory element (e.g.,160). Additionally, or alternatively, the activated218 coordinator may generate230 and communicate234 commands to compensate for or correct the source of the error. For example, if the progress information indicates that processing in a first module or system portion is occurring slower than anticipated or directed by the commands, updated commands may be sent to a second module or system portion indicating a delayed start time for processing at that module or system portion. Alternatively, commands may be generated230 and communicated234 to the first module or system portion directing it to process a task or workpiece at a faster rate.
• The activated218 coordinator may also act as a clearinghouse for progress information. Such a service may serve to conserve communication bandwidth in some systems. For example, referring toFIG. 3, a second embodiment ormethod310 for coordinating controllers in a distributed control system includes determining214 a task, activating218 a coordinator, identifying222 subtasks, identifying226 controllers, generating230 commands and communicating234 the commands to the controllers as described above. Additionally, thesecond method310 includes identifying314 information sources for providing subtask progress information, collecting318 the subtask progress information from the information sources and communicating322 the subtask progress information to the controllers as appropriate to the subtasks.
• Identifying314 information sources proceeds in a manner similar to identifying226 controllers for performing the subtask. System configuration and capabilities information is received from a memory or database or is provided atactivation218 time. The location and capabilities of available sensors (e.g.,124,126,128,132) and controllers (e.g.,108,110,112,114,116) are evaluated in light of the identified126 subtasks. If information from a controller or sensor that is not immediately available to an active controller (e.g.,114,110) would be of benefit to the active controller in completing a subtask, that information source is identified314. For example, information from the third set ofsensors128 may indicate that a workpiece is about to become available to thefourth system portion146. Therefore, thecoordinator170 may request that information from one or more of the third set ofsensors128 be delivered to thefirst coordinator170. The first coordinator may then relay322 the delivered information to thefourth controller114.
• Collecting318 the subtask progress information may include receiving the information from information sources after requesting or subscribing to the information from the identified314 information sources. Again, the information may be received from a network or through other communication means.
• Communicating322 the subtask progress information to the controllers as appropriate to the subtasks may include communicating information only when it would be useful to a particular controller. In some embodiments, the progress information is communicated only to those controllers for which the information would be useful. This may reduce network traffic.
• Identifying222 subtasks, identifying226 controllers, generating230 commands, and communicating234 the commands can be performed one time for each coordinator activation. Communicated234 commands can include start times. Therefore, subtasks that need to occur in the future can be requested in advance. Alternatively, as indicated by the dashed line inFIG. 2, identifying222 subtasks, identifying226 controllers, generating230 commands, communicating234 the commands and, where included, monitoring238 (e.g.,314,318), can occur on a repeated, iterative, or on-going basis as the task or workpiece sequentially requires the services of a new or next system portion (e.g.,140-148). In a further alternative, the identifying222,226 and generating230 commands may occur at an initial phase and the communicating234 of the commands may occur as the task or workpiece requires the services of the new or next system portion. Furthermore, generated230 commands may be revised or replaced as needed.
• Referring toFIG. 4, an embodiment of adocument processing system404 includes asupervisory element408, afirst marking engine410, asecond marking engine412 and atransportation system414.
• For example, the first and second markingengines410,412 may be xerographic marking engines. Alternatively, one or more marking engines of an embodiment may be of other technologies, such as, but not limited to, ink jet marking technology.
• Thetransportation system414 transports print media such as afirst sheet416 and asecond sheet418 between thefirst marking engine410 and thesecond marking engine412. In the illustratedsystem404, the transportation system includes a plurality of transport modules. For instance, the plurality of transport modules includes a first, second, third, fourth, fifth, sixth andseventh transport module420,422,424,426,428,430,432. Thesystem404 may include additional modules. For example, the additional modules may include a media orpaper feeder433, which delivers sheets of print media or paper to one or both of the markingengines410,412. Additional modules (not shown) may transport print media from either or both markingengines410,412 to other devices, including, but not limited to, additional marking engines and/or output devices such as paper trays, stackers, collators, staplers and other binders. Furthermore, the plurality of transport modules may form paths that branch off from the illustrated path (420,422,418,424,426,430,432) to transport sheets to other marking engines (not shown) or other devices.
• In the illustrateddocument processing system404, each transport module420-432 includes transport actuators. For example, the transport modules420-432 include motor drivennips434 for driving or urging print media through thetransport system414. Additionally, or alternatively, the modules420-432 may include flippers or gates for redirecting print media toward other portions (not shown) of thetransportation system414. Furthermore, the modules may include other kinds of transport actuators. For instance, air jets and/or spherical nips may be included in the transport modules (e.g.,420-432). For the purposes of illustration, the modules ofFIG. 4 are associated with thenips434 depicted to their left. The transport modules420-432 of thedocument processor system404 include sensors. For instance, the sensors may be sheet presence or position sensors. As illustrated, each module420-432 includes aleft side sensor436 and aright side sensor438.
• Each transport module420-432 also includes or is associated with arespective module controller440,442,444,446,448,450,452. For example, the module controllers440-452 control the actions of the transport actuators of their respective modules420-432 and receive and relay information from theirrespective sensors436,438.
• Thesupervisory element408 is operative to generate (e.g.,214) sheet processing task descriptions or itineraries describing respective sheet processing tasks, to activate (e.g.,218) respective sheet coordinators (e.g., afirst sheet coordinator460 and a second sheet coordinator464) and to communicate the respective sheet processing task descriptions to the respective sheet coordinators (e.g.,460,464). For example, thesupervisory element408 receives ajob description470. Thejob description470 may include descriptions of sheets or pages. The descriptions may include images, or references to images stored elsewhere and indications as to an order in which the images are to appear on sheets of print media. For example, thejob description470 includes page description language describing text and fonts and graphic items as well as their location on particular pages of a document. Thesupervisory element408 activates, instantiates or spawns (e.g.,218) a sheet coordinator for each sheet or page (a sheet may have two sides and may, therefore, comprise two pages). Thesupervisory element408 analyses thejob description470 and may schedule or plan operations to create the document described in thejob description470. In so doing, thesupervisory element408 generates respective sheet processing task descriptions or itineraries for the transportation and processing of sheets between system resources.
• For instance, regarding the transportation of a sheet between system resources, an example itinerary or sheet processing task description may have the following form:
• Itin 1 1 11
• feeder1 feed 19.544
• me1 print image27 20.201
• m1 left2right 23.341
• m2 left2right 23.495
• m3 left2right 23.625
• m4 left2right 23.755
• m5 left2right 23.885
• m6 left2right 24.015
• m7 left2right 24.145
• me2 print image28 24.275
• finisher1 stack 27.415
• The first line is, for example, an itinerary or sheet processing task description identifier. The rest of the itinerary specifies, for example, that a component named feeder1 (e.g.,433) should feed a sheet at time 19.544, then a component named me1 should execute a print action on an image named image27 at a later time, then a component named m1 should execute an action (move the sheet left to right) at a later time, and so on.
• The respective sheet coordinators (e.g.,460,464) are operative to receive the respective sheet processing task descriptions or itineraries and, based on those respective descriptions, identify (e.g.,222) a plurality of respective sheet processing subtasks to be performed in order to complete the respective sheet processing tasks, identify (e.g.,226) respective controllers for controlling respective process actuators to perform the respective sheet processing subtasks, generate (e.g.,230) respective commands for performing the respective sheet processing subtasks and communicating (e.g.,234) the respective commands to the respective module controllers as appropriate to the respective subtasks. Additionally, the respective sheet coordinators (e.g.,460,464) may identify (e.g.,314) respective information sources that are able to provide progress information regarding the performance of the respective subtasks, collect (e.g.,318) the respective progress information from the respective subsets of information sources and communicate (e.g.,322) the respective progress information to the respective module controllers as appropriate to the respective sheet processing subtasks.
• For example, the information sources may include thesensors436,438. Additionally, or alternatively, the module controllers themselves may maintain models or estimators of the progress of respective subtasks. Such models are referred to as sheet observer models. In this regard, the module controllers or the estimates or models of the module controllers may be considered information sources.
• For instance, in the illustrateddocument processing embodiment404, subtasks for a first sheet may have included matching a speed ofnips434 of thefirst module420 to a speed of a sheet exiting thefirst marking engine410 and receiving thefirst sheet416 therefrom. A second subtask might have been fornips434 of thesecond module422 to match the speed of thefirst sheet416 as it exited thefirst module420. A subtask of thethird module424 may have been to match the speed of thefirst sheet416 as a leading edge thereof exited thesecond module422. Yet another subtask may have been for thenips434 of the first, second andthird modules420,422,424 to accelerate or to begin to accelerate thefirst sheet416 to ahigher transportation system414 transport speed.
• Additional subtasks associated with the fourth, fifth andsixth modules426,428,430 may have included matching associated nip434 speeds to the speed of thefirst sheet416 as it entered eachmodule426,428,430 and/or continuing to accelerate thesheet416.
• The transfer or movement of a sheet from module to module must be done in a coordinated manner. In thedocument processing embodiment404, themodules410,412,420-433 are tightly coupled by their relationship to a sheet. For example, at any given point in time, a plurality of modules may be in contact with the same sheet. If thenips434 of modules contacting a sheet are driven at different speeds or with different rates of acceleration or deceleration, the sheet (e.g.,416,418) may be damaged or distorted in a manner that causes a jam in thetransportation system414 orsystem404 as a whole. The sheet coordinators (e.g.,460,464) ensure cooperative or coordinated actuation of the actuators or modules (e.g.,410,412,420-433). For example, at the instant depicted inFIG. 4, thefirst sheet416 is in contact with portions of the fourth, fifth andsixth modules426,428,430. Thefirst sheet coordinator460 is shown in communication with the fourth, fifth, sixth and seventh module controllers446-452. For example, thefirst sheet coordinator460 may be sending commands to the fifth andsixth module controllers448,450 that result in the fifth andsixth modules428,430 driving thefirst sheet416 in a cooperative manner. For instance, the fifth andsixth module controllers448,450 may be directed to begin decelerating thefirst sheet416. Additionally, thefirst sheet coordinator460 may be requesting or receiving sensor information or sheet observer model information from thefourth module controller446. For instance, thefirst sheet coordinator460 may be requesting to be notified when a trailing edge of thefirst sheet416 passes theleft sensor436 of the fourth module. Additionally, thefirst sheet coordinator460 may be asking or receiving sensor information from thesixth module430. For instance, thefirst sheet coordinator460 may be requesting to be notified when a leading edge of thefirst sheet416 passes or enters a field of view of theright sensor438 of the sixth module.
• This sensor information may be relayed by the sheet coordinator to theseventh module controller452. Additionally, or alternatively, thefirst sheet coordinator460 may update a model, such as a world observer model of the task or of the subtasks based on the information from the information sources or sensors (e.g.,436,438).
• In addition to possibly relaying sensor information, thefirst sheet coordinator460 may be sending commands directing theseventh module controller452 to prepare theseventh module432 to receive thefirst sheet416. For instance, theseventh module controller452 may be directed to drivenips434 of theseventh module432 at a speed compatible with the speed of thefirst sheet416 as the leading edge thereof exits thesixth module430. Additionally, the fifth, sixth andseventh module controllers448,450,452 may be receiving commands directing that they begin decelerating the first sheet in preparation for its entry into thesecond marking engine412. Thefirst sheet coordinator460 may also be transmitting commands to thefourth module controller446 releasing it from service or subtasks related to the transportation of thefirst sheet416. Alternatively, prior commands may have included an expiration event, such as a time limit or sensor reading, the occurrence of which automatically deactivates or releases the fourth module controller from services related to thefirst sheet416.
• At a point later in time than the instant depicted inFIG. 4, thefirst sheet416 may enter thesecond marking engine412 for processing. For example, the second marking engine may be used to print an image on a second side of the first sheet or may apply color markings that thefirst marking engine410 did not apply. At that later point in time, the first sheet will no longer be in contact with thefourth module426 and the trailing edge of the first sheet will be about to exit thefifth module428. Thefourth module controller446 may have already been released (as described above) from subtasks associated with processing thefirst sheet416 and may have begun performing subtasks associated with processing thesecond sheet418. Thefifth module controller448 may be about to be similarly released.
• The sixth andseventh transport modules430,432 and the second marking engine (or module)412 are likely all in contact with thefirst sheet416. Therefore, thefirst coordinator460 is generating or has generated (e.g.,230) and will communicate or has communicated (e.g.,234) commands for the sixth andseventh transport modules430,432 and thesecond marking engine412 or a markingengine module controller474. The commands may be cooperative in nature. For example, thetransport modules430,432 may be directed to slow the sheet to a speed compatible with capabilities of the markingengine412. Additionally, commands for the secondmarking engine controller474 may direct it to control thesecond marking engine412 to accept the first sheet at the compatible speed and to place specified marks on portions of thefirst sheet416. As thefirst sheet416 continues into thesecond marking engine412, the sixth andseventh module controllers450,452 will be released from subtasks associated with thefirst sheet416, or deactivated. Eventually, thefirst sheet416 will exit the second marking engine ormodule412 and be delivered to other modules (e.g., transport modules, finishers, stackers and/or other print engines). The first coordinator will continue to send appropriate commands to the subsequent modules, relay progress information and release or deactivate controllers, in the sequential manner described above, until the task described in the task description, or itinerary, received when thefirst coordinator460 was activated218 is completed. When the task is completed, thefirst coordinator460 may be deactivated.
• Similar processing occurs with regard to the second464 and subsequent (not shown) coordinators and second418 and subsequent (not shown) sheets. For example, as depicted inFIG. 4, thesecond sheet418 is within the first, second andthird modules420,422,424. Thesecond sheet coordinator464 is depicted as in communication with the first, second, third and fourth module controllers440-446. For example, thesecond sheet coordinator464 may be directing the second andthird module controllers442,444 to drive thesecond sheet418 at the same speed and/or with the same acceleration, receiving or requesting sensor information from thesensors436,438 of thefirst module420 and/or thethird module424, releasing thefirst module controller440 from tasks associated with transporting thesecond sheet418, and/or directing thefourth module controller446 to prepare to receive thesecond sheet418 by drivingnips434 of thefourth module426 at a speed appropriate to, or compatible with, a speed of thesecond sheet418, as a leading edge thereof exits thethird module424 and enters thefourth module426. As thesheets416,418 are transported through thesystem404, the sheet coordinators deactivate or release module controllers no longer processing their respective sheets and send commands to downstream controllers preparing them to receive their respective sheets.
• Prior to the moment depicted inFIG. 4., the first coordinator generated230 and sent234 commands to afeeder module controller478 directing it to control thefeeder433 to deliver thefirst sheet416 to the first marking engine (or transport modules on a path thereto (not shown)) and may have generated230 and sent234 commands to a firstmarking module controller482 instructing it to control thefirst marking engine410 to place particular marks on portions of thefirst sheet416 and deliver thefirst sheet416 to thefirst transport module420. As mentioned above, when their respective tasks, as described in their respective sheet processing task descriptions or itineraries, are completed, the respective sheet coordinators (e.g.,460,464) are deactivated. For instance, they are de-instantiated or placed in an idle mode to await re-initialization with information from a new sheet processing task description.
• Supervisory element and module controller embodiments may be made substantially in software stored in computer storage devices, such as memory elements, and run by computational platforms, such as microprocessors, microcontrollers, and digital signal processors. Alternatively, supervisory elements and module controllers may be embodied in various combinations of hardware and software.
• In a prototype, the transport module controllers were each embodied in separate computational platforms associated with transport modules on a one-to-one basis. Each transport module included a plurality of nips and flippers. Marking engines are known to include their own controllers. Thesupervisory element408 was run on one computational platform and activated or spawned sheet coordinators (e.g.,460,464) were software elements run on another computational platform. However, embodiments wherein the sheet coordinators, or activating data associated with the sheet coordinators, move from module controller to module controller (e.g.,440-452,474-482) as their respective sheets move from module to module (e.g.,433,410,420-432,412), are contemplated.
• It will be appreciated that various of the above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into many other different systems or applications. Also that various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.

Claims (25)

1. A method for coordinating controllers in a distributed control system that is operative to perform a plurality of simultaneous tasks, the method comprising:

determining a respective task of the plurality to be performed;

activating a respective coordinator in association with the respective task, the respective coordinator then:

identifying a plurality of respective subtasks to be performed in order to complete the respective task;

identifying, based on the identified respective subtasks, a plurality of respective controllers, for controlling a plurality of actuators to perform the respective subtasks;

generating respective commands for directing the performance of the respective plurality of subtasks;

communicating the respective commands to the respective controllers as appropriate to the respective subtasks;

collecting respective progress information from at least one respective information source; and

communicating the respective progress information to the respective controllers as appropriate to the respective subtasks.

2. The method ofclaim 1 further comprising:

deactivating respective controllers as respective subtasks are determined to be completed based on the collected respective progress information, thereby releasing the respective controllers to execute commands communicated from another coordinator associated with another task.

3. The method ofclaim 1 further comprising:

deactivating the coordinator when the respective task is determined to be completed.

4. The method ofclaim 1 further comprising:

maintaining at least one model of the respective subtasks; and

deactivating at least one of the coordinator and the respective controllers when the respective subtasks are completed according to the at least one model.

5. The method ofclaim 1 further comprising:

identifying at least one respective information source that is able to provide progress information regarding a performance of the respective subtasks.

6. The method ofclaim 5 wherein identifying at least one respective information source comprises:

at least one of identifying at least one feedback variable from at least one controller and identifying at least one sensor.

7. The method ofclaim 1 further comprising:

maintaining at least one of a command model of the respective task based on the communicated respective commands and a progress model of the respective task based on the collected respective progress information.

8. The method ofclaim 1 further comprising;

maintaining a command model of the respective task based on the communicated respective commands;

maintaining a progress model of the respective task based on the collected respective progress information;

comparing a first indicated status of the respective task from the progress model to a second indicated status from the command model, thereby generating a respective task progress error value; and

comparing the respective task progress error value to a respective task progress error limit; and

performing an error task if the respective task progress error value is beyond the respective task progress error limit.

9. The method ofclaim 8 wherein performing the respective error task comprises:

generating at least one of a respective error message identifying the respective task and describing the respective task progress error and at least one updated command message for correcting or compensating for a respective task progress error associated with the respective task progress error value.

10. A system comprising:

at least one supervisory element that is operative to generate respective task descriptions describing respective tasks, to activate respective coordinators to be responsible for orchestrating a completion of the respective tasks and to communicate the respective task descriptions to the respective coordinators;

a plurality of controllers that are operative to control a plurality of actuators according to respective subtask descriptions received from the respective coordinators; and

a plurality of information sources that are operative to report status information regarding respective progress of respective tasks to the respective coordinators;

wherein the respective coordinators are operative to receive the respective task descriptions, identify, based on the respective task descriptions, a plurality of respective subtasks to be performed in order to complete the task, identify, based on the respective subtasks, a subset of the plurality of respective controllers, for controlling a subset of the plurality of actuators to perform the respective subtasks, generate respective commands for performing the respective plurality of subtasks, communicate the respective commands to the respective controllers as appropriate to the respective subtasks, collect the respective progress information from the respective subsets of information sources and communicate the respective progress information to the respective controllers as appropriate to the respective subtasks.

11. The system ofclaim 10 wherein the respective coordinators are further operative to monitor the respective progress of the respective subtasks.

12. The system ofclaim 11 wherein the respective coordinators are further operative to deactivate the respective controllers as the subtasks are completed.

13. The system ofclaim 10 wherein the respective coordinators are further operative to report the respective progress of the respective tasks to the supervisor and wherein the supervisor is further operative to deactivate the respective coordinators when the respective tasks are completed.

14. The system ofclaim 10 wherein the at least one supervisory element is operative to activate the respective coordinators by spawning or instantiating the respective coordinators.

15. The system ofclaim 10 wherein the supervisor is operative to activate the respective coordinators by communicating respective task descriptions to respective idle coordinators.

16. The system ofclaim 10 wherein the plurality of controllers comprises:

at least one of a transport actuator controller, a nip controller, a feeder controller, finisher controller and a marking engine controller.

17. The system ofclaim 10 wherein the plurality of actuators comprises:

at least one of a xerographic marking engine actuator, a transport actuator, a nip driver, a feeder actuator and a finisher actuator.

18. The system ofclaim 10 wherein the respective coordinators are operative to identify respective subsets of the plurality of information sources that are able to provide progress information regarding the performance of the respective subtasks by at least one of identifying at least one sensor and identifying at least one feedback variable from at least one controller of the plurality of controllers.

19. The system ofclaim 11 wherein the respective coordinators are operative to monitor the respective progress of the respective subtasks by at least one of maintaining a command model of the task based on the communicated respective commands and maintaining a progress model of the task based on the respective progress information.

20. The system ofclaim 11 wherein the respective coordinators are operative to monitor the respective progress of the respective subtasks by maintaining respective models of the respective tasks based on the communicated respective commands and maintaining respective progress models of the respective tasks based on the respective progress information and wherein the respective coordinators are further operative to compare respective first indicated statuses of the respective tasks from the progress model to respective second indicated statuses of the respective tasks from the command based model, thereby generating respective task progress error values, to compare the respective task progress error values to respective task progress error limits and to perform respective error tasks if the respective task progress error values are beyond the respective error limits.

21. A method for processing a sheet in a document processing system, the method comprising:

receiving a sheet description specifying a document processing job to be performed on the sheet;

activating a coordinator as a proxy for the sheet, the coordinator then:

identifying a plurality of respective subtasks to be performed in order to process the sheet according to the sheet description;

identifying, based on the identified respective subtasks, a plurality of respective controllers for controlling a plurality of actuators to perform the respective subtasks;

generating respective commands for directing the respective plurality of subtasks;

communicating the respective commands to the respective controllers as appropriate to the respective subtasks;

collecting respective progress information from at least one respective information source;

communicating the respective progress information to the respective controllers as appropriate to the respective subtasks;

estimating the respective progress of the respective subtasks;

deactivating the respective controllers as the subtasks are completed; and

deactivating the coordinator when the task is completed.

22. A document processing system comprising:

a first xerographic marking engine;

a transportation system that is operative to transport sheets of print media to and/or from the first engine, the transportation system including a plurality of transport actuators,

at least one supervisory element that is operative to generate respective sheet processing task descriptions describing respective sheet processing tasks, to activate respective sheet coordinators to be responsible for orchestrating the respective sheet processing tasks according to the respective sheet processing task descriptions;

a plurality of respective transport controllers that are operative to control respective sets of transport actuators, of the plurality of transport actuators, according to respective sheet processing subtask descriptions received from the respective sheet coordinators;

a first marking module controller that is operative to control aspects of processing of the first marking engine according to respective sheet processing subtask descriptions received from the respective sheet coordinators, and

a plurality of information sources that are operative to report status information regarding respective progress of respective sheet processing tasks to the respective sheet coordinators;

wherein the respective sheet coordinators are operative to receive the respective sheet processing task descriptions, identify, based on the respective sheet processing task descriptions, a plurality of respective sheet processing subtasks to be performed in order to complete the respective sheet processing tasks, identify, based on the respective sheet processing subtasks, a subset of the plurality of respective transport controllers and the first marking module controller, for controlling a subset of the plurality of transport actuators and the first marking engine to perform the respective sheet processing subtasks, identify respective subsets of the plurality of information sources that are able to provide progress information regarding the performance of the respective sheet processing subtasks, generate respective commands for performing the respective plurality of sheet processing subtasks, communicate the respective commands to the respective transport controllers and/or first marking engine controller as appropriate to the respective subtasks, collect the respective progress information from the respective subsets of information sources and communicate the respective progress information to the respective transportation controllers and/or first marking engine controller as appropriate to the respective sheet processing subtasks.

23. The document processing system ofclaim 22 wherein the plurality of transport controllers comprises:

at least one of a nip controller, an air jet controller, a flipper controller and a spherical nip controller.

24. The document processing system ofclaim 22 further comprising:

at least a second marking engine;

at least a second marking module controller that is operative to control aspects of processing of the at least a second marking engine according to respective sheet processing subtask descriptions received from the respective sheet coordinators,

wherein the transportation system is further operative to transport sheets of print media to and/or from the at least a second marking engine; and

wherein the respective sheet coordinators are further operative to identify, based on the respective sheet processing subtasks, at least a respective one of the at least a second marking module controller, for controlling at least a respective one of the at least a second marking engine to perform the respective sheet processing subtasks, and communicate the respective commands to at least a respective one of the at least a second marking module controller as appropriate to the respective subtasks.

25. A method for coordinating controllers in a distributed control system that is operative to simultaneously perform operations on a plurality of workpieces, the method comprising:

activating a respective coordinator for each respective workpiece, wherein each respective coordinator encompasses knowledge regarding an itinerary of operations to be applied to each respective workpiece, each respective coordinator thereby:

issuing respective commands to a series of actuator controllers for directing a series of actuators to perform respective operations on the respective workpiece according to the itinerary;

maintaining at least one respective estimate of progress of the respective operations performed on the respective workpiece; and

deactivating the respective coordinator when the itinerary of operations is completed.

Images