Disclosure of Invention
The invention aims to overcome the defects in the prior art, provides a PCIe equipment multiplexing system in a multi-node system, and solves the technical problems of low efficiency, software process server loss and the like caused by the fact that a multi-node server needs to be powered on and powered off when switching nodes.
In order to achieve the above purpose, the invention is realized by adopting the following technical scheme:
the invention provides a PCIe equipment multiplexing system in a multi-node system, which comprises a BMC module, at least two node modules, at least one CPLD module and at least two multiplexers; the node modules are internally provided with PCIe controllers, the PCIe controllers output parallel hot plug signals and bus data signals, and the parallel hot plug signals are connected to the CPLD module and used for interacting with the CPLD module; the bus data signals corresponding to the node modules are respectively connected to the input end of one multiplexer, the clock signals of the node modules are respectively connected to the input end of the other multiplexer, and the output ends of the multiplexers are respectively connected to the BMC modules; and the CPLD module and the BMC module are also connected with GPIO control signals, and are used for notifying the multiplexer to switch the channels through the BMC module and notifying the node module to switch through the CPLD module, and the CPLD module generates the SEL selection signals according to the GPIO control signals and sends the SEL selection signals to the control end of the multiplexer.
Optionally, the parallel hot plug signals include a lock switch signal, a key switch signal, a power enable signal, a power ready signal, a clock enable control signal, and a PCIe reset signal.
Optionally, if the number of the node modules is two, the multiplexer adopts a two-way multiplexer, the two input ends, the output end and the control end of the two-way multiplexer are respectively marked as an end A, an end B, an end C and an end SEL, and when the SEL selection signal received by the end SEL is at a low level, the end C is connected and conducted with the end A; when the SEL selection signal received by the SEL terminal is at a high level, the C terminal is connected to the B terminal.
Optionally, the GPIO control signal is generated by a tact switch or a BMC module.
Optionally, the PCIe reset signal of the BMC module is connected to the CPLD module, if the number of the node modules is two, when one of the two node modules is in a normal working state, the PCIe reset signal of the BMC module is at a high level, otherwise, is at a low level.
Optionally, if the PCIe controller is a serial hot plug signal bus, the IO expansion chip is connected to the serial hot plug signal bus, and a parallel hot plug signal is output through the IO expansion chip.
Compared with the prior art, the invention has the beneficial effects that:
according to the PCIe equipment multiplexing system in the multi-node system, after the server is electrified, the BMC module is set by default to be connected with one node module, and after the server is electrified, the node module can exchange data information with the BMC module normally; in the system operation, if another node module requests to exchange data information with the BMC module, the CPLD module is used for carrying out heat removal of the current node, after the heat removal is completed and the heat loading is carried out on the request node, the PCIe bus data path is switched through the multiplexer, and the current node is switched to the request node. All the steps do not need to restart the server after power failure, and the use of other software in the node is not affected, so that the reliability and the availability of the server are improved.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.
Embodiment one:
the embodiment of the invention provides a PCIe equipment multiplexing system in a multi-node system, which comprises a BMC module, at least two node modules, at least one CPLD module and at least two multiplexers; the node modules are internally provided with PCIe controllers, and the PCIe controllers output parallel hot plug signals and bus data signals, and the parallel hot plug signals are connected to the CPLD module and used for interacting with the CPLD module; the bus data signals corresponding to the node modules are respectively connected to the input end of one multiplexer, the clock signals of the node modules are respectively connected to the input end of the other multiplexer, and the output ends of the multiplexers are respectively connected to the BMC module; and the CPLD module and the BMC module are also connected with GPIO control signals, and the CPLD module is used for informing the multiplexer of carrying out channel switching through the BMC module and informing the node module of carrying out switching through the CPLD module, generating an SEL selection signal according to the GPIO control signals and sending the SEL selection signal to the control end of the multiplexer.
As shown in fig. 1, one BMC module is provided, two node modules are provided, one CPLD module is provided, and two multiplexers are provided:
the two node modules are respectively denoted as node 1 and node 2, and the parallel hot plug signal of node 1 includes a lock switch signal hp1_mrl#, a key switch signal hp1_button#, a power enable signal hp1_pwren, a power ready signal hp1_pwr_good, a clock enable control signal hp1_clken_ # and a PCIe reset signal hp1_perst#; the parallel hot plug signals of node 2 include a lock switch signal hp2_mrl#, a key switch signal hp2_button#, a power supply enable signal hp2_pwren, a power supply ready signal hp2_pwr_good, a clock enable control signal hp2_clken_ # and a PCIe reset signal hp2_perst#; the bus data signal of the node 1 is PCIE1_TX/RX, and the bus data signal of the node 2 is PCIE2_TX/RX.
The two multiplexers are respectively a double-path multiplexer and are marked as a multiplexer 1 and a multiplexer 2, the two input end, the output end and the control end of each multiplexer are respectively marked as an A end, a B end, a C end and a SEL end, and when the SEL selection signal received by the SEL end is at a low level, the C end is connected and conducted with the A end; when the SEL selection signal received by the SEL terminal is at a high level, the C terminal is connected and conducted with the B terminal; the bus data signals PCIE1_TX/RX of the node 1 and PCIE2_TX/RX of the node 2 are respectively connected to an A end and a B end of the multiplexer 1, and a C end of the multiplexer 1 is connected to a bus data end of the BMC module; the clock signal CLK1 of the node 1 and the clock signal CLK2 of the node 2 are respectively connected to the a terminal and the B terminal of the multiplexer 2, and the C terminal of the multiplexer 2 is connected to the clock terminal of the BMC module; the SEL terminals of multiplexer 1 and multiplexer 2 are both connected to the CPLD module. The CPLD module receives the GPIO control signal, and when the BMC module is connected with the node 1, the CPLD module sets the SEL signal to be low level; when the BMC module is connected to node 2, the CPLD module sets the SEL signal high.
The GPIO control signal is generated by a touch switch or a BMC module, for example, one end of the touch switch is connected with a power supply, the other end of the touch switch is connected with the BMC module and the CPLD module, a high-level signal is generated when the touch switch is pressed down, and a low-level signal is generated when the touch switch is released; the BMC module can also directly generate a high-level signal as a GPIO control signal to be connected to the CPLD module. For example, the node 1 is connected with a BMC module, and when the level of the GPIO control signal changes, the CPLD module informs the node 1 and the BMC module to remove and unload related programs; related programs are accessed and loaded through the node 2 and the BMC module.
And the PCIe reset signal of the BMC module is connected to the CPLD module, and is high level when one of the two node modules is in a normal working state, or is low level.
The working principle of this embodiment is as follows:
1. the server is powered on, and the BMC module is mounted on the node 1 by default as follows:
(1) The CPLD will node 1: the HP1_MRL#, HP1_PRSNT# signals are set to a low level, HP1_BUTTON# is set to a high level, HP1_PWR_GOOD is set to a low level, the HP2_MRL#, HP2_PRSNT#, HP2_BUTTON# signals of node 2 are set to a high level, and HP2_PWR_GOOD is set to a low level;
(2) The PCIe controller of node 1 turns on the HP1 PWREN signal, setting to high level;
(3) The CPLD receives the high level HP1 PWREN signal, delays for a period of time (t 1) and sets the HP1 PWR GOOD signal to be high level; and sets the path selection signal SEL of the multiplexer to a low level, i.e., the port C and the port a are connected;
(4) After receiving the high level signal of HP1 PWR GOOD, the PCIe controller of the node 1 sets HP1 CLKEN# to be low level, and generates a CLK1 clock; and after a delay of a certain period of time (t 2), setting HP1_PERST# to a high level;
(5) The node 1 and the BMC module start PCIe channel training and complete loading of the BMC module into the software system;
2. the CPLD module sets the HP1_BUTTON# signal to be at a low level for a period of time (t 3) after the BMC receives the change through the change of the GPIO signal or the change of a switch, and the node 1 generates an interrupt to inform the software to thermally remove the BMC module.
The steps of node 1 hot removing the BMC module are as follows:
(1) After receiving the removal notification, the PCIe controller of the node 1 uninstalls the corresponding software;
(2) The PCIe controller of node 1 sets signal HP1_ perst# low;
(3) After a delay period (t 4), the PCIe controller of node 1 sets signal HP1 clken# high;
(4) After a further delay for a period of time (t 5), the PCIe controller of node 1 asserts signal HP1 PWREN low;
(5) The CPLD receives the high-level HP1 PWREN low-level signal and sets the HP1 PWR GOOD as a low signal;
(6) The CPLD will node 1: the HP1_MRL# and HP1_PRSNT# signals are set high;
3. after the BMC module is thermally removed from the node 1, the BMC module is thermally loaded at the node 2 as follows:
(1) The CPLD will node 2: the HP2_MRL# and HP2_PRSNT# signals are set to low, the HP2_PWR_GOOD is set to low, and the HP2_BUTTON# signal is set to high;
(2) After a delay period (t 6), the CPLD sets HP2_BUTTON# to a low level for a period of time;
(3) The PCIe controller of node 2 turns on the HP2 PWREN signal, setting high;
(4) The CPLD receives the high level HP2 PWREN signal, delays for a period of time (t 7) and sets the HP2 PWR GOOD signal to be high level; and sets the path selection signal SEL of the multiplexer to a high level, i.e., the port C and the port B are connected;
(5) After receiving the high level signal of HP2 PWR GOOD, the PCIe controller of the node 2 sets HP2 CLKEN# to be low level, so that CLK2 clock is generated; and after a delay of a certain period of time (t 8), setting HP2_PERST# to a high level;
(6) The node 2 and the BMC module start PCIe channel training, and the node 2 finishes loading the BMC module into the software system;
4. the BMC module completes loading in node 2 and maintains the current state until the next switch command arrives.
Embodiment two:
as shown in fig. 2, on the basis of the first embodiment, if the PCIe controller is a serial hot plug signal bus, an IO expansion chip is connected to the serial hot plug signal bus, and a parallel hot plug signal is output through the IO expansion chip.
Embodiment III:
as shown in fig. 3, the BMC module is provided with one, the node module is provided with three, the CPLD module is provided with one, the multiplexers are provided with two, the three node modules are respectively marked as node 1, node 2 and node 3, the two multiplexers are respectively marked as a three-way multiplexer, and are marked as multiplexer 1 and multiplexer 2, and three input ends, output ends and two control ends of each multiplexer are respectively marked as an a end, a B end, a D end, a C end, a SEL0 end and a SEL1 end, as shown in table 1:
table 1: node selection table
| SEL1 | SEL0 | Node |
| Low level | Low level | Node 1 |
| Low level | High level | Node 2 |
| High level | Low level | Node 3 |
When SEL0 is low level and SEL1 is low level, the C end is connected and conducted with the A end, and node 1 is selected; when SEL0 is high level and SEL1 is low level, the C end is connected and conducted with the B end, and node 2 is selected; when SEL0 is at a low level and SEL1 is at a high level, the C end is connected and conducted with the D end, and a node 3 is selected; the bus data signals PCIE1_TX/RX of the node 1, PCIE2_TX/RX of the node 2 and PCIE3_TX/RX of the node 3 are respectively connected to an A end, a B end and a D end of the multiplexer 1, and a C end of the multiplexer 1 is connected to a bus data end of the BMC module; the clock signal CLK1 of the node 1, the clock signal CLK2 of the node 2 and the clock signal CLK3 of the node 3 are respectively connected to the a terminal, the B terminal and the D terminal of the multiplexer 2, and the C terminal of the multiplexer 2 is connected to the clock terminal of the BMC module; the SEL0 and SEL1 terminals of the multiplexers 1 and 2 are connected to the CPLD module. The CPLD module receives GPIO control signals (the switch is continuously pressed for several times, or the GPIO continuously outputs a plurality of pulse signals, which represents the switching of the BMC module to a node module which can be set according to actual needs), and when the BMC module is connected with the node 1, the CPLD module sets the SEL0 signal as a low level and the SEL1 signal as a low level; when the BMC module is connected with the node 2, the CPLD module sets the SEL0 signal to be high level and the SEL1 signal to be low level; when the BMC module is connected to node 3, the CPLD module sets the SEL0 signal to low and the SEL1 signal to high.
Embodiment four:
as shown in fig. 4, one BMC module is provided, three node modules are provided, two CPLD modules are provided, two multiplexers are provided, the three node modules are respectively denoted as node 1, node 2 and node 3, and the two multiplexers are respectively denoted as a multiplexer 1 and a multiplexer 2; the two CPLD modules are respectively marked as CPLD0 and CPLD1, the CPLD0 and the CPLD1 are communicated by signal connection, and the CPLD0 informs the CPLD1 to unload or load the BMC module in the node 3; when CPLD0 sends ENABLE# to be low pulse, CPLD1 unloads or loads the BMC module in node 3, and when unloading or loading is completed, COMPLETE# outputs a low pulse signal.
According to the PCIe equipment multiplexing system in the multi-node system, the hot plug signal of the PCIe controller of the multi-node is connected with the CPLD module, and the CPLD module controls the BMC module to control the hot removal or hot loading of PCIe equipment in different nodes. In the switching process, the connected hot plug signal changes without manual intervention, and the change is completed by a program preset by the CPLD module; the CPLD module controls the path selection signal of the multiplexer, switches PCIe bus data signals of a plurality of nodes and selectively connects one of the PCIe bus data signals; each port of the multiplexer is respectively connected with PCIe bus data signals of a plurality of nodes, and after being selected, the ports are connected with PCIe bus data signals of the BMC module; each port of the multiplexer is respectively connected with clock signals of multiple nodes, and after the clock signals are selected, the clock signals are connected with CLK (clock signal) of the BMC module, so that clock homology of the PCIe bus can be ensured; the switching process does not need the power-off operation of the server and restarting the operating system, and the switching of the connection between the BMC module and a plurality of nodes is finished on line.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that modifications and variations could be made by those skilled in the art without departing from the technical principles of the present invention, and such modifications and variations should also be regarded as being within the scope of the invention.