BACKGROUNDThis invention relates generally to data centers, and more particularly to efficient cooling of computing devices within a data center.
Heat removal is a prominent factor in computer system and data center design. The number of servers deployed in a data center has steadily increased while the increase in server performance has increased the heat generated by the electronic components in the servers during operation. Because the reliability of servers used by the data center decreases if they are permitted to operate at a high temperature over time, a portion of the data center's power is used for cooling electronics in the servers. As the number or servers included in a data center increases, a greater portion of the power consumed by the data center is used to cool electronics within the server.
Conventionally, the servers in the data center are individually equipped with a cooling system to dissipate heat produced during operation. Commonly, each server includes a fan to dissipate heat generated by the server during operation. However, these internal fans generally consume about 10%-15% of the power used by the servers, and they also produce heat during operation, thereby limiting the ability of these fans to dissipate heat.
SUMMARYEmbodiments of the invention reduce or eliminate the need for internal fans to cool servers in a data center, at least under normal operating conditions. In one embodiment, a data center includes a cold aisle that receives cold air, where the cold aisle is adjacent to one side of a set of servers. A hot aisle adjacent to another side of the servers has a pressure less than the pressure of the cold aisle. This pressure difference between the cold aisle and the hot aisle causes cold air to flow from the cold aisle through the server to the hot aisle, thereby cooling the electronic components in the servers (and heating the air flow). In an embodiment, a cold air supply unit that is external to the servers, such as a fan, supplies the cold air to the cold aisle from a cooling system and causes the pressure difference. Additionally, the hot aisle may include one or more exhaust units that are external to the servers. The exhaust units circulate air from the hot aisle and thus help produce the pressure difference that causes the circulation of the air through the servers. The heated air from the hot aisle may be cooled and then recirculated through the cold aisle, or the cool air may be obtained elsewhere, such as ambient air.
In one embodiment, a sensor monitors air temperature or air flow proximate to the server and adjusts the flow of cold air into the cold aisle accordingly. For example, if the temperature near the server reaches a threshold value or the airflow near the server reaches a threshold flow rate, the sensor communicates with a control system, which produces a control signal increasing the rate at which cold air is supplied to the cold aisle or modifying the direction in which cold air flows into the cold aisle. In another embodiment, the server includes an internal fan and an internal fan control system. The internal fan remains inactive until the fan control system determines that the temperature within the server equals a threshold temperature and activates the internal fan to augment the airflow through the server.
BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is an overhead view of a data center for cooling servers without relying on internal fans in accordance with an embodiment of the invention.
FIG. 2 is a side view of a data center for cooling servers without relying on internal fans showing airflow throughout the data center in accordance with an embodiment of the invention.
FIG. 3A is a front view of an example server in accordance with an embodiment of the invention.
FIG. 3B is an overhead view of components within an example server in accordance with an embodiment of the invention.
The figures depict various embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.
DETAILED DESCRIPTIONData Center Architecture
Anexample data center100 cooling one ormore servers105 is illustrated inFIG. 1. In one embodiment, acold aisle110 is adjacent to a first side of apartition102 and ahot aisle120 is adjacent to a second side of thepartition102. In an embodiment, thepartition102 includes one ormore servers105 oriented so that a first side of the one ormore servers105 is adjacent to thecold aisle110 and a second side of the one ormore servers105 is adjacent to thehot aisle120. Thecold aisle110 includes acold air supply115 while, in an embodiment, thehot aisle120 includes one ormore exhaust units125. Additionally, one ormore sensors117 are proximate to aserver105, are included in thecold aisle110 and/or are included in thehot aisle120.
Thepartition102 includes one or more openings though which air is able to flow. In an embodiment, thepartition102 comprises a rack or other structure to which one or more devices, such as one ormore servers105 or other electronic devices, may be attached. For example, the one ormore servers105 are mounted to one or more racks and may have different sizes, such as 1.5-2 rack units (“U”). Thepartition102 is designed to increase airflow through theservers105 that are included within thepartition102. For example, thepartition102 includes a server rack that is designed to increase the amount of air directed through theservers105 included in the rack.
Aserver105 has one or more input openings on a first side and one or more output openings on a second side adjacent. Aserver105 is oriented so the one or more input openings are adjacent to thecold aisle110 and the one or more output openings are adjacent to thehot aisle120. Air from thecold aisle110 enters theserver105 via the one or more input openings, travels through theserver105 and exits the server through the one or more output openings into thehot aisle120. Hence, the input and output openings allow air to travel through theserver105 to cool components included in theserver105. Anexample server105 is further described below in conjunction withFIGS. 3A and 3B.
Cold air is supplied to thecold aisle110 from acold air supply115, such as a large fan or other air distribution device. In an embodiment, thecold air supply115 is coupled to a cooling system, further described below in conjunction withFIG. 2. As used herein, “cold air” may refer to air having a temperature less than an ambient air temperature, air having a temperature below a specified temperature, or air having a lower relative temperature than air in a different region. For example, air included in thecold aisle110, referred to as “cold air,” has a first temperature, while air included in thehot aisle120, referred to has “hot air,” has a second temperature that is higher than the first temperature. In different embodiments, the position of thecold air supply115 relative to thecold aisle110 may differ. For example, thecold air supply115 may be positioned above, below, or to the side of thecold aisle110. Additionally, in some embodiments, multiplecold air supplies115 provide cold air to thecold aisle110 and may have different positions relative to thecold aisle110. For example,cold air supplies115 are positioned above and below or below and to the side of thecold aisle110. For purposes of illustration,FIG. 1 shows an implementation with acold air supply110 positioned above thecold aisle110. As a result of thecold air supply115, thecold aisle110 has a higher pressure than ahot aisle120, and this pressure difference causes cold air to flow from the higher pressurecold aisle110 through the one or more input openings of aserver105 or thepartition102 to the lower pressurehot aisle120.
In an embodiment, thepartition102 is configured so that air flow paths external to theservers105 are substantially blocked such that the airflow path of least resistance from thecold aisle110 to thehot aisle120 is through theservers105. Configuring thepartition102 so that the airflow path of least resistance is through theservers105 allows moreefficient server105 cooling by increasing the amount of air passing through theservers105. In another embodiment, thepartition102 blocks substantially all airflow from thecold aisle110 to thehot aisle120 except for the airflow through theservers105, so that substantially all of the airflow from thecold aisle110 to thehot aisle120 is through theservers105. To facilitate airflow from thecold aisle110 to the hot aisle, in one embodiment thecold aisle110 may be pressurized while thehot aisle120 is depressurized to facilitate airflow from thecold aisle110 to thehot aisle120. As the cold air passes through theserver105, it flows over components within theserver105, dissipating heat generated from operation of the electric components in theservers105.
In different embodiments, thecold air supply115 may statically or dynamically control the amount of air supplied to thecold aisle110 to modify the airflow through theservers105. In an embodiment where the air supply is statically controlled, thecold air supply115 is louver-based and supplies cold air in different directions, at different flow rates, and/or at different temperature levels. In an alternative embodiment, thecold air supply115 dynamically modifies the airflow supplied to thecold aisle110 by changing the speed of one or more supply fans, repositioning one or more air supply louvers (or otherwise redirecting the airflow), or changing the temperature to which the airflow is cooled. Modifying the supply fan speed, supply louver position, and/or air temperature allows thecold air supply115 to more suitably cool theservers105 included in thepartition102. Hence, implementations of thecold air supply115 allow non-uniform air flow and/or air temperature throughout thecold aisle110, enabling different locations within thecold aisle110, such as locations proximate todifferent servers105, to have a different air flow rate and/or a different air temperature. Additionally, the air flow from thecold air supply115 may be determined or modified based on the size of theservers105 being cooled.
After flowing through theservers105, cold air enters thehot aisle120 because it has a lower pressure than thecold aisle110. Because the air extracts heat from components within one ormore servers105, when passing from thecold aisle110 to thehot aisle120, the air temperature increases so that air in thehot aisle120 has a higher temperature than air in thecold aisle110. In an embodiment, thehot aisle120 includes one ormore exhaust units125, such as exhaust fans, which extract air from thehot aisle120. WhileFIG. 1 shows an examplehot aisle120 with twoexhaust units125, in other embodiments, the hot aisle may include a different number ofexhaust units125. In an embodiment, theexhaust unit125 is coupled to a cooling system, further described below in conjunction withFIG. 2, so that air flows from thehot aisle120 into the one ormore exhaust units125 and into the cooling system, where it is cooled and recirculated into thecold aisle110 via thecold air supply115. Alternatively, cold air enters thehot aisle120 and is directed outside of thedata center100.
Thedata center100 also includes one ormore sensors117 in locations where air flows from thecold aisle110 to thehot aisle120. Thesensors117 monitor air flow, air temperature, air humidity, absolute air pressure, differential air pressure, or any other data that describes air flow or air temperature, and combinations thereof. In an embodiment, thesensors117 are placed in locations where airflow is likely to be less than other locations, such as a ceiling or a wall where thepartition102 abuts another surface, so that the temperature of the sensor locations is likely to be higher than other locations. For example,sensors117 are placed in corners of thecold aisle110 to monitor airflow through the corners, the temperature of the corners, the pressure difference between thecold aisle110 and thehot aisle120 or another value characterizing air flow through the sensor location. In another embodiment,sensors117 are positioned at locations within thecold aisle110, at locations within thehot aisle120, at locations within one ormore servers105 or in any combination of the above-described locations.
Thesensors117 communicate with acontrol system116 coupled to, or included in, the cooling system and/or thecold air supply115 to modify how air is cooled by the cooling system or how cold air is supplied to thecold aisle110 by thecold air supply115. Thecontrol system116 generates a control signal responsive to data from one ormore sensors117 to modify operation of the cooling system and/or thecold air supply115. For example, responsive to detecting a temperature reaching a threshold value, an air flow reaching a threshold flow rate, or a pressure difference between thecold aisle110 and thehot aisle120 falling below a threshold value, asensor117 communicates with thecontrol system116, which generates a control signal increasing the rate at which thecold air supply115 supplied to thecold aisle110 or modifying the direction in which cold air is supplied to thecold aisle110 by thecold air supply115. Hence, thesensors117 andcontrol system116 implement a feedback loop allowing thedata center100 to modify how cold air flows through theservers105 responsive to changes in the data center environment, improving the cooling efficiency.
FIG. 2 is a side view of the airflow indata center100 that is capable of cooling theservers105 without depending on fans within theservers105, according to one embodiment. The arrows indicate the flow of air throughout thedata center100. As illustrated, acooling system210 is coupled to acold air supply115 and to anexhaust unit125. WhileFIG. 2 shows a singlecold air supply115 and asingle exhaust unit125, other embodiments may have multiple cold air supplies115 and/ormultiple exhaust units125.
Thecooling system210 comprises a Heating, Ventilating and Air Conditioning (“HVAC”) system, which extracts heat from air. For example, thecooling system210 uses free-air cooling, such as air-side or liquid-side economization to cool the air. In an embodiment, thecooling system210 includessecondary cooling systems212, such as an evaporative cooling system, an absorption cooling system, an adsorption cooling system, a vapor-compression cooling system, or another cooling system to extract heat from air. In another embodiment, thecooling system210 also modifies the humidity of the cool air to improve reliability and/or longevity of theservers105 being cooled. For example, thecooling system210 produces cold air having a humidity within a specified range, such as 20% to 60% humidity, to thecold aisle110.
In one embodiment, thecooling system210 receives heat from theexhaust units125 included in thehot aisle120, cools and dehumidifies the received air, and supplies the cooled and dehumidified air to thecold air supply115, where it is supplied to thecold aisle110. In this embodiment, thecooling system210 is aclosed system210, which recirculates air from thehot aisle120 to thecold aisle110 once the air is cooled and dehumidified. As illustrated by the arrows inFIG. 2, the cooled air travels from thecooling system210 to thecold air supply115, which supplies the cold air to thecold aisle110. In an embodiment, thecold air supply115 comprises one or more fans or one or more air nozzles, one or more air jets, or other device for directing air flow.
Cooled air from thecold air supply115 enters thecold aisle110. Because thecold aisle110 has a higher pressure than thehot aisle120, and thepartition102 includes one or more openings for air flow, the cold air flows from thecold aisle110 to the lower pressurehot aisle120. To flow from thecold aisle110 to thehot aisle120, the cold air passes through the openings in thepartition102, so that the cold air is drawn through thepartition102. In an embodiment, thepartition102 includes one ormore servers105 that have one or more input openings on a first side adjacent to thecold aisle110 and one or more output openings on a second side adjacent to thehot aisle120. The input openings allow cold air to enter theserver105, travel through theserver105, flowing over components within theserver105. After traveling through theserver105, the output openings enable air to exit theserver105 into thehot aisle120.
As cool air travels through thepartition102 and/or aserver105 from thecold aisle110 to thehot aisle120, a portion of the air travels across, or through, one ormore sensors117 which monitor attributes of the airflow, such as air temperature, air humidity, absolute air pressure of thecold aisle110 or of thehot aisle120, or a pressure difference between thecold aisle110 and thehot aisle120. Thesensors117 communicate the monitored attributes to acontrol system116, which is coupled to or included in, thecold air supply115 or thecooling system210. Thecontrol system116 generates a control signal modifying operation of thecold air supply115 and/or thecooling system210 to modify the cold air supplied to thecold aisle110. For example, responsive to asensor117 detecting a temperature above a threshold value, an air flow below a threshold flow rate or a pressure difference between thecold aisle110 and thehot aisle120 falling below a threshold value, thecontrol system116 generates a control signal increasing the rate or direction at which thecold air supply115 supplies cold air to thecold aisle110 or generates a control signal directing cold air from thecold air supply115 towards certain areas in thecold aisle110 needing increased cooling. For example, the control signal causes thecold air supply115 to more cold air towards a region of thepartition102 where asensor117 indicates a temperature above a threshold value or an airflow rate below a threshold value. Alternatively, thecontrol system116 generates a control signal causing thecooling system210 to further reduce the temperature of the air provided to thecold aisle110.
One ormore exhaust units125 are included in thehot aisle120 to extract air from thehot aisle120 and to direct air from thehot aisle120 to thecooling system210, where the air is again cooled. Hence, the one ormore exhaust units125 implement a closed-loop where air is cooled by thecooling system210 and recirculated to thecold aisle110 via thecold air supply115. Because the pressure differential betweencold aisle110 andhot aisle120 causes air to flow through thepartition102, and electronic devices included in thepartition102, electronic devices included in thedata center100 are cooled without relying on air moving devices, such as fans, operating at individual electronic devices. Additionally, reducing the use of locally-implemented air moving devices reduces power consumption of the electronic devices, making thedata center100 more power efficient.
Server Design
To use the airflow from thecold aisle110 to thehot aisle120 in an efficient manner, aserver105 may internally channel airflow to cool components within theserver105. Components within theserver105 may be oriented so that the airflow is increased over components with higher operating temperatures. In an embodiment, the server includes additional physical elements, such as one or more air dams, that redirect airflow within theserver105 to increase airflow over components with higher operating temperatures.FIG. 3A is a front view of one embodiment of aserver105 illustrating input openings305 for cold air. In one embodiment, theserver105 includesmultiple input openings305A,305B,305C,305D,305E to increase the amount of air flowing through theserver105. For example, theserver105 is positioned on thepartition102 so that theinput openings305A-305E are adjacent to thecold aisle110. Because of the pressure difference between thecold aisle110 and thehot aisle120, air flows from the higher pressurecold aisle110 through theinput openings305A-305E to reach thehot aisle120.
The placement of components within theserver105 and the amount of heat generated by the components within theserver105 and/or desired maximum temperature of components within theserver105 may also be used to determine the air flow to be supplied by thedata center100 for cooling. For example, the placement and operating temperature of components within theserver105 is used when determining the temperature, flow rate or directionality, or cold air supplied to thecold aisle110 by thecold air supply115. In an embodiment, asensor117 included in aserver105 monitors the temperature of one or more components within theserver105 and communicates the temperature to acontrol system116 which generates a control signal modifying the cold air provided to thecold aisle110 by thecold air supply115.
FIG. 3B is an overhead view of components within one embodiment of aserver105 for use in a data center in accordance with an embodiment of the invention. Components within theserver105 are arranged so that a high percentage of the air flowing through theserver105 flows across the components having higher operating temperature. In an embodiment, theserver105 includes one ormore processors310A,310B, an input/output hub320, one or morehard drives330A,330B aninternal fan350 and afan control system355 which receive power form apower supply unit340. The components shown inFIG. 3B are examples, and in other embodiments theserver105 includes additional and/or different components.
Theprocessors310A,310B and the input/output hub320 are proximate to theinput openings305A-305C so that cold air flows across the one ormore processors310A,310B and the input/output hub230 to cool the components. In an embodiment, one or more heat sinks are coupled to theprocessors310A,310B to increase the surface area over which heat generated by a processor310 is radiated. Cold air entering from one or more of theinput openings305A-305C flows over the one or more processors310 and the input/output hub320 to carry generated heat away from the one or more processors310 and the input/output hub320. Movement of the cold air carries the heat out of the interior of theserver105 through the one ormore output openings360A,360B,360C. In one embodiment, the one ormore output openings360A-360C output air into ahot aisle120.
In an embodiment, thehard drives330A,330B are positioned within the interior of theserver105 to help direct the airflow across components having a higher operating temperature, such as theprocessors310A,310B and input/output hub320. Additionally, the interior of theserver105 may include additional components to direct the airflow from theinput openings305A-306C through the interior of theserver105.
In another embodiment, theserver105 does not include aninternal fan350 is configured so that airflow through theserver105 from thecold aisle110 through theserver105 to thehot aisle102 cools components within theserver105. In this configuration, movement of cold air from the cold aisle through theserver105 carries heat out of the interior of theserver105 through the out ormore output openings360A,360B,360C. In an alternative embodiment, theserver105 may also include aninternal fan350 and an internalfan control system355 to provide a redundant cooling system for theserver105. Theinternal fan350 is generally in a power-off state is activated responsive to a command from the internalfan control system355. In an embodiment, the internalfan control system355 monitors the temperature of one or more components, such as a processor310 or a motherboard, inside theserver105. Responsive to the temperature of one or more of the monitored components equaling a threshold value, the internalfan control system355 activates the internalfan control system355 to supplement the airflow through theserver105 and further cool components within theserver105. Alternatively, the internalfan control system355 determines the air pressure near the input openings305 and the air pressure near the output openings360 and activates the internalfan control system355 responsive to the pressure difference or responsive to the air pressure near the input openings305 or near the output openings360. In another implementation, the internalfan control system355 communicates with one ormore sensors117 outside of theserver105 and activates theinternal fan350 responsive to receiving data from the one ormore sensors117 about external air pressure or temperature external to theserver105. Hence, theinternal fan350 provides theserver105 with an auxiliary cooling device supplementing airflow from thecold aisle110 through theserver105 to thehot aisle120. This may be useful, for example, if the externally driven airflow is not sufficient or not cold enough to cool the components within theserver105.
SUMMARYThe foregoing description of the embodiments of the invention has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Persons skilled in the relevant art can appreciate that many modifications and variations are possible in light of the above disclosure.
Some portions of this description describe the embodiments of the invention in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.
Any of the steps, operations, or processes described herein may be performed or implemented with one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented with a computer program product comprising a computer-readable medium containing computer program code, which can be executed by a computer processor for performing any or all of the steps, operations, or processes described.
Embodiments of the invention may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a tangible computer readable storage medium, which include any type of tangible media suitable for storing electronic instructions, and coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.
Embodiments of the invention may also relate to a computer data signal embodied in a carrier wave, where the computer data signal includes any embodiment of a computer program product or other data combination described herein. The computer data signal is a product that is presented in a tangible medium or carrier wave and modulated or otherwise encoded in the carrier wave, which is tangible, and transmitted according to any suitable transmission method.
Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based hereon. Accordingly, the disclosure of the embodiments of the invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.