A single high-density AI training rack can draw more than 100 kW. Uptime Institute puts the NVIDIA GB200 NVL72 at roughly 132 kW in a 19-inch frame — about 100 kW to cold plates, with more than 25 kW still rejected to air.
The challenge does not end at the cold plate. That heat still has to cross two or three fluid circuits, several heat exchangers, and a heat rejection plant before it reaches the atmosphere, and every transfer costs temperature difference.
TL;DR
-
Air runs out around 40–50 kW per rack. ASHRAE puts airflow for such a rack at up to 5,000 cfm, with fan power alone reaching at least 5 kW in a 50 kW rack.
-
The CDU is a chemistry boundary, not just a hydraulic one. ASHRAE holds technology cooling chloride under 5 ppm against under 50 ppm for facility water.
-
Plate exchangers are the usual CDU selection. Liquid-to-liquid units are rated at 4–5°C (7–9°F) approach; the liquid-to-air equivalent at 15°C (27°F).
-
Coolant temperature is spreading both ways. ASHRAE extended its technology cooling classes down to S-20, while NVIDIA designs its Rubin-generation rack loop to a 45°C supply point.
On this page: the density problem · the heat path · why the loops are separated · facility heat rejection · plate versus shell-and-tube · materials and water chemistry · reliability and future load · common mistakes · what to specify · FAQs
Why AI density moved cooling out of the server room
Air cooling did not stop working. It ran out of air. Beyond roughly 40–50 kW per rack, the airflow and fan power needed to move that heat stop being practical.
ASHRAE — the American Society of Heating, Refrigerating and Air-Conditioning Engineers — publishes widely used technical guidance for data center thermal environments through its Technical Committee 9.9. TC 9.9 put numbers on the boundary in Emergence and Expansion of Liquid Cooling in Mainstream Data Centers. A 40 to 50 kW rack “could be up to 5000 cfm” of airflow, and “in a 50 kW rack, the fan power translates to be at least 5 kW.” That is ten percent of rack power spent moving air, before the chiller plant does anything.
The installed base sits well below that line. Uptime Institute reports modal rack densities shifting toward 10 kW, and puts the consensus for where direct liquid cooling becomes necessary or economically justified at 20–30 kW per rack.
AI clusters skip past that range. The ASHRAE, NEMA, and PNNL AI Data Center Energy Performance Framework describes GPU clusters driving “40–100 kW per rack vs. 5–10 kW in legacy CPU racks.” It calls for a technology cooling system wherever densities “routinely exceed 50–120 kW per rack.”
Scale changes the character of the problem. Lawrence Berkeley National Laboratory’s 2025 update puts 2024 U.S. data center electricity use at 192 TWh, reaching 649 TWh by 2030 in its reference case. Cooling systems and power distribution losses accounted for 31% of the 2024 total.
At those numbers, cooling stops being a room-conditioning exercise. It becomes a process heat-transfer system with a distributed load, a water balance, and a heat sink.
The heat path: TCS, FWS, and the exchanger between them
Heat leaving an AI processor follows a fixed chain: die → cold plate → technology cooling system → CDU heat exchanger → facility water system → heat rejection → atmosphere or reuse. Each interface costs temperature difference.
Die to cold plate. The processor rejects heat through its lid into a cold plate with fine internal channels. Uptime reports Intel processors specified for liquid cooling with case temperatures as low as 57°C (134.6°F). DRAM degrades above 85°C (185°F). Those two ceilings set everything upstream.
Cold plate to technology cooling system. The TCS is the IT-side circuit: cold plates, rack manifolds, quick disconnects, and the piping serving a defined set of hardware. ASHRAE’s Water-Cooled Servers describes it as “data-center-specific and most often associated with a specific set of IT hardware.”
TCS to facility water system through the CDU. The coolant distribution unit holds the liquid-to-liquid heat exchanger that moves heat between the circuits, plus pumps, filtration, instrumentation, and controls. The same white paper lists six functions for it: transfer heat from TCS to FWS, circulate TCS coolant, permit a coolant other than water, prevent condensation, hold a coolant chemistry different from the FWS, and supply flexible coolant temperature.
That fourth function drives more CDU control logic than any efficiency target. The CDU “is responsible for monitoring the ambient dew point and elevating the secondary water loop (TCS) supply temperature to at least 3.6°F (2°C) above the room dew point to prevent condensation.” Condensate inside energized IT equipment is not a performance issue. It is a damage event. Designs that couple IT equipment directly to facility water inherit the same obligation at a loop far harder to hold above dew point.
FWS to heat rejection. The facility water system is the building or campus circuit — “often a campus- or site-wide building system,” in that paper’s phrasing — running to chillers, towers, fluid coolers, or dry coolers.
Two conventions catch process engineers out. In ASHRAE’s usage, the primary side of the CDU exchanger is the facility water side and the secondary side is the technology cooling side. And two class systems govern different loops: W-classes describe facility water supply temperature, S-classes describe TCS supply temperature.
W-classes and S-classes: what each one governs
The facility water classes carry their upper limit in the name — W17, W27, W32, W40, W45, and W+ — each with a common minimum of 2°C (35.6°F). W32 means a facility water supply maximum of 89.6°F (32°C). The reference card for ASHRAE’s Thermal Guidelines for Data Processing Environments lists a cooling tower rather than a chiller as the primary equipment from W32 upward.
S-classes are newer. ASHRAE TC 9.9 rewrote its Liquid Cooling Guidelines for Datacom Equipment Centers to introduce “a new set of environmental classes (S Classes) for the supply temperature of the Technology Cooling System (TCS) coolant.”
The low end of that set was extended afterward. TC 9.9 members on the ASHRAE Journal Podcast in June 2025 said the range “which was S-30 to S-50 has expanded to S-20 to S-50,” with S-20 corresponding to 68°F (20°C).
Read that as a widening spread, not a one-way trend. Some architectures need colder coolant at the chip; others are designed deliberately warm, and the warm end is where the facility economics live. The CDU exchanger sits between those objectives, and its approach temperature reconciles them. Uptime observes facility water temperatures “seem to be converging around 32°C (89.6°F)” in practice.
Why the loops are separated: chemistry, not just hydraulics
Most explanations of the CDU stop at pressure isolation. That is real, and it is the smaller half of the answer.
ASHRAE states the larger half plainly. The liquid-to-liquid heat exchanger between TCS and FWS “is able to provide a critical benefit to ITE: separation of water quality control domains.”
| Representative parameter | Facility water system (FWS) | Technology cooling system (TCS) |
|---|---|---|
| Chloride | <50 ppm | <5 ppm |
| Sulfate | <100 ppm | <10 ppm |
| Total hardness (as CaCO₃) | <200 ppm | <20 ppm |
| Bacteria | <1000 CFU/mL | <100 CFU/mL |
| Biocide | — | Required |
Selected parameters from ASHRAE TC 9.9, Water-Cooled Servers (2019), which also specifies pH, conductivity, suspended solids, sulfide, turbidity, and corrosion inhibitor requirements for both loops. Consult the full publication for the complete specification.
The TCS is held roughly an order of magnitude tighter for two reasons. It wets microchannel cold plates with clearances measured in fractions of a millimeter, and a pinhole there lands water on energized silicon.
Filtration splits the same way. ASHRAE recommends an absolute TCS filter rating “7 to 10 times smaller than the finest channel dimension in the IT cooling equipment.” The facility strainer guideline is “35 to 40 mesh (or 500 to 400 μm).” Vertiv’s CoolChip CDU data sheet reflects the split: 500 µm primary, 25 or 50 µm secondary.
Treat the two loops as separate chemistry domains and the equipment questions organize themselves. Wetted materials, corrosion allowance, cleanability, and inspection access all get answered differently on either side of the same exchanger.
Facility heat rejection: where the heat actually leaves
Downstream of the CDU, the equipment is conventional process cooling hardware running an unconventional duty cycle. Selection turns on two variables: how close the facility water supply must approach ambient, and how much water the site can spend getting there.
| Rejection method | Approaches | Typical role at AI density | Water use | Principal limitation |
|---|---|---|---|---|
| Open cooling tower + heat exchanger | Wet bulb | Warm-water loops (W32 and above) | Highest | Treatment, filtration, Legionella risk management |
| Closed-circuit fluid cooler | Wet bulb (wet mode) | Keeps process fluid clean in one package | High in wet mode | Larger footprint per ton than open tower |
| Adiabatic / hybrid cooler | Between dry and wet bulb | Water-constrained sites | Up to 90% reduction vs. evaporative | Scale growth on pre-cooling media |
| Dry cooler | Dry bulb | Cool climates, warm-water loops | None | Dry-bulb approach limits design-day capacity |
| Water-cooled chiller | Mechanical, ambient-independent | Trim, low-S-class TCS, residual air load | Via tower | Compressor energy |
| Waterside economizer | Wet bulb, bypassing compressors | Shoulder and winter hours | Tower water | Needs an exchanger and a supply temperature it can satisfy |
One distinction controls the whole table. An open cooling tower approaches the ambient wet bulb; a dry cooler approaches the dry bulb. That sets the achievable facility water temperature on a design day, and therefore the temperature available to the CDU.
SPX’s Cooling Tower Fundamentals defines approach as the difference between cold water temperature and entering wet bulb. It adds a piece of commercial reality: “it is not customary in the cooling tower industry to guarantee any approach of less than 5°F.”
Warm-water operation buys free cooling hours, and the payoff is climate-specific. Uptime Institute reports systems running on free cooling “50% to 80% or more of the time.” A 2025 Cooling Technology Institute paper reports water reduction “up to 90%” for an adiabatic cooler against an evaporative tower, while naming scale growth on the pre-cooling media as the failure mode to design against.
Why warmer coolant widens the dry-cooling window
Usable coolant temperature at the rack buys ambient margin at the heat rejection plant. That is why the warm end of the range matters commercially.
NVIDIA’s DSX reference documentation states that “at the rack, the TCS is designed for a 45°C supply design point.” That point, it adds, “can widen the opportunity for dry cooling and reduce cooling overhead when site and ambient conditions permit.” Note the loop: 45°C (113°F) is technology cooling supply to the cold plates, not facility water to the CDU.
The qualifier carries the engineering. A high TCS supply point raises the facility water temperature the CDU can work with, extending the hours a dry cooler carries the load. Whether it does at a given site depends on ambient conditions, CDU approach, exchanger sizing, and redundancy. NVIDIA’s own documentation concedes that an Arizona site “may need trim support for much of the year and mechanical cooling during the hottest conditions.”
Warm coolant is an opportunity to design for, not a guarantee of chiller-free operation.
Liquid cooling is not the same as waterless cooling
A closed IT-side loop says nothing about site water consumption. The heat rejection architecture decides that.
Uptime Institute puts it plainly: “water is consumed only in the heat rejection loop.” A closed collection loop “consumes little (if any) water — even when serving DLC equipment.” LBNL’s 2024 energy report makes the point from the other side, describing an IT liquid cooling system on a waterside economizer as one that “consumes substantial amounts of water, like all evaporative cooling systems.”
Uptime’s guidance: water-cooled rejection is typically most energy-efficient where water is plentiful, while water-stressed sites “should prioritize dry free cooling and mechanical cooling modes.” Going dry spends fan energy, compressor energy on design days, and plot space instead of water. The cold plate decides none of it.
Heat reuse, and why the grade matters
Liquid cooling raises the temperature grade of recovered heat, which is what makes reuse worth considering. LBNL-affiliated authors report as much in a 2026 National Academy of Engineering article, Avoiding Waste Heat through AI Infrastructure Thermal Integration. Advanced cooling raises the temperature of dissipated heat “from about 30°C (typical of air cooling) to more than 50°C,” roughly 86°F to more than 122°F.
Whether that heat gets reused is economic rather than thermal. It takes an offtaker within reach and a supply agreement both parties can carry, and the interface between them is another heat exchanger.
Plate versus shell-and-tube: what actually decides the selection
Plate technology is typically the preferred selection inside a liquid-to-liquid CDU. Shell-and-tube construction becomes a candidate further out in the facility infrastructure, and only where specific conditions justify it. This comparison is often written to reach a conclusion. It should be written to reach a selection basis.
Why plate exchangers suit the CDU duty
The CDU duty is a very close approach in a very small envelope, with clean, treated fluid on both sides. ASHRAE describes the exchanger as “typically a flat plate construction with nominal plate spacing ranging from 0.079 to 0.315 in. (2 to 8 mm).”
Commercial liquid-to-liquid CDUs are rated at 4–5°C (7–9°F) approach. Vertiv publishes 4°C ATD across its 600 kW, 1,350 kW, and 2,300 kW models; CoolIT publishes 5°C ATD for its row-based CHx units. Corrugated plates deliver high turbulence and high surface area at low hold-up volume, which is what that specification demands. Tubular construction is not a practical way to hit that approach in that footprint.
The sharpest contrast sits within plate technology itself. CoolIT rates its liquid-to-air AHx240 at 15°C (27°F) approach — roughly three times the approach, for rejecting to air instead of water. The fluid on the cold side matters more than the exchanger geometry.
Where shell-and-tube becomes a candidate
The selection opens up on the facility side and at the plant interfaces, where duty, fluid quality, and service life change the weighting.
| Selection factor | Favors compact plate | Favors shell-and-tube |
|---|---|---|
| Approach temperature | Close approach in minimal volume | Larger approach acceptable |
| Footprint and hold-up volume | Severely constrained | Space available |
| Fluid cleanliness | Treated, filtered, both sides | Open-loop, silt-laden, or biologically active water |
| Fouling and cleaning | Chemical clean-in-place; gasketed units open for access | Mechanical cleaning of straight tubes; removable bundle |
| Pressure and temperature | Within the gasket or braze envelope | High pressure, high temperature, or wide differential |
| Inspection | Plate pack inspection on gasketed units | Bundle pull, eddy current, tube-by-tube examination |
| Repair philosophy | Replace plates and gaskets | Plug, retube, or replace the bundle |
| Code and mechanical duty | Standard catalog envelopes | ASME Section VIII stamped, custom nozzles, unusual geometry |
| Materials | Plate alloys, gasket elastomers | Full range of alloys, clad construction, no elastomeric limit |
Two constraints deserve naming precisely. Gasketed plate units are bounded by the gasket — Kelvion publishes design pressures of roughly 11 to 33 barg (160 to 480 psig) across its standard gasketed series, with a welded-side variant to 64 barg (930 psig). Brazed plate units reach higher pressures but cannot be opened for mechanical cleaning — a real constraint wherever the cold side carries open-loop tower water.
The CDU exchanger and the facility interface are two different equipment problems that happen to sit twenty feet apart. Compactness governs one. Fouling, cleanability, mechanical robustness, and inspection access govern the other. Selection follows the duty, not a preference for a technology.
Materials and water chemistry: when an upgrade is justified
Material upgrades in cooling water service are justified by a documented corrosion mechanism, not by a general ranking of alloys.
Open and closed loops fail differently. A closed facility loop with a corrosion inhibitor and controlled makeup can often run 300-series stainless, or carbon steel, successfully. The outcome depends on water chemistry, temperature, the treatment program, and system design rather than on the alloy alone.
An open tower loop concentrates dissolved solids by evaporation and drives chloride upward with every cycle of concentration. A makeup water analysis is not a loop water analysis.
Chloride, temperature, and crevice geometry together drive pitting and stress corrosion cracking in austenitic stainless. Where those conditions are credible, duplex and 6% molybdenum grades carry substantially higher pitting resistance than 304 or 316. One trap catches technically literate readers: residual chlorine and chloride are different specifications, routinely conflated, driving different mechanisms.
Mixed metallurgy is the failure nobody owns. ASHRAE lists copper, low-zinc brasses, and passivated stainless as acceptable in both loops, and carbon steels in the facility loop only. Aluminum is conditional — “normally avoided in liquid cooling loops because of its potential as a galvanic corrosion catalyst of other metals,” though usable “provided there is proper surface treatment of the metal.”
Cold plates, manifolds, CDU internals, and facility piping are usually bought separately, so galvanic couples form at the interfaces between packages. The material is permissible; the coupling is what has to be engineered.
Coolant choice is part of the design basis. A 25% propylene glycol solution appears throughout current AI deployment guidance; NVIDIA’s Data Center Best Practices with DGX B300 directs sites to have PG25 handling procedures in place where applicable, rather than mandating a composition. Glycol buys freeze protection and biological control at the cost of heat-transfer performance and pump head. Book those penalties at the start.
Most facility cooling water does not justify titanium, and most closed technology cooling loops do not justify anything beyond properly passivated stainless. Upgrades earn their cost where a documented mechanism exists — seawater or brackish makeup, high-chloride reclaimed water, aggressive treatment chemistry. Everywhere else, the money is better spent on treatment, filtration, and access for cleaning. The selection logic for corrosion-resistant alloys is the same in cooling water as in chemical service: match the alloy to the mechanism.
Reliability, maintainability, and the load that has not arrived yet
An air-cooled hall carries a large buffer of conditioned air. A liquid-cooled rack does not, and that missing thermal mass changes the redundancy problem.
Thermal ride-through drops from minutes to seconds
Uptime Institute is direct about the consequence. Facilities below 10 kW per rack historically had “several, sometimes tens of, minutes” of ride-through. Cold plate systems “cannot tolerate loss of circulation for more than a few seconds.” ASHRAE TC 9.9 members made the same point on the record: lose flow to even a medium-power processor and “you’re talking seconds of time before you will have adverse thermal effects.”
That changes what redundancy has to accomplish. Uptime notes that “acceptable ride-through time typically requires UPS power for CDU pumps, at a minimum.” Generator start time is no longer inside the thermal budget.
Uptime’s Tier definitions frame the rest. Tier III adds concurrent maintainability — “no shutdowns when equipment needs maintenance or replacement.” Tier IV adds fault tolerance and requires continuous cooling. Uptime’s paper on continuous cooling goes further, recommending it “at densities beyond 4 kilowatts (kW)/rack, regardless of Tier” — a threshold today’s AI racks exceed by more than an order of magnitude.
For fabricated equipment in the heat path, the practical requirements follow from concurrent maintainability:
-
Isolation valves and bypasses at every exchanger, so a unit can be removed without draining the loop
-
Pull space for a tube bundle, or plate-pack access clearance, designed into the layout rather than discovered during the first cleaning
-
Vent, drain, and sample connections placed for real maintenance sequences and chemistry monitoring
-
Instrumentation on both sides — inlet and outlet temperature, differential pressure — so fouling is trended rather than inferred after a capacity shortfall
-
Redundant capacity sized honestly — an N+1 exchanger that cannot be isolated is not actually N+1
Facility thermal infrastructure outlives the IT it serves
Servers and CDUs turn over on a hardware cycle. Facility piping, heat exchangers, pumps, electrical distribution, and heat rejection equipment do not.
The gap is widening. NVIDIA’s DGX SuperPOD reference architecture for GB200 NVL72 specifies 1.2 MW thermal design power across an eight-rack scalable unit, roughly 150 kW per rack. Its Rubin-generation documentation puts rack-level power at approximately 225 kW. The ASHRAE, NEMA, and PNNL framework states the designer’s problem directly. Balance “meeting current needs and factoring anticipated growth in computational complexity, energy consumption, and thermal loads.”
So facility-side equipment should be evaluated against plausible future load, not only the first deployment. That means deliberate provisions rather than blanket oversizing — headers sized for a later flow increase, exchanger surface and nozzles that accept a re-rate, space for an added unit, and hydraulic headroom for a future bundle’s pressure drop.
Oversizing alone is not the answer. An exchanger far off its design point fouls differently, controls poorly at low load, and costs more than it needs to. The framework’s own recommendation is modular “pay-as-you-grow” expansion in repeatable blocks.
Common mistakes in data center cooling equipment specification
Six specification errors recur often enough to be worth naming.
-
Treating the CDU as a hydraulic device. Its principal contribution is chemistry separation. Design the two loops as separate water quality domains, or the cold plates inherit the facility loop’s chloride, hardness, and particulate.
-
Spending the whole approach budget in one place. Case temperature, TCS supply, CDU approach, facility water, tower approach, and ambient wet bulb form one chain. A generous margin at one link is usually borrowed from another.
-
Selecting facility-side equipment on CDU logic. Compactness governs inside the cabinet. Fouling, cleanability, mechanical robustness, and inspection access govern where open-loop water, silt, and biological activity are present.
-
Ignoring mixed metallurgy across procurement packages. Cold plates, manifolds, CDUs, and facility piping are often bought separately. Galvanic couples form at the interfaces nobody owns.
-
Treating liquid cooling as waterless cooling. A closed IT loop can still reject heat through an evaporative tower. Water consumption is set at the heat rejection plant.
-
Sizing the facility side for the first IT deployment only. Piping, exchangers, and heat rejection outlive several hardware generations.
Specify the loop, the water, and the access
The most useful thing a project brings to a cooling equipment inquiry is not a heat exchanger type. It is a description of the duty complete enough that a fabricator or OEM can select one:
-
Which loop the unit serves, and who controls its chemistry
-
Design and operating pressure and temperature on both sides
-
The approach the plant depends on
-
A real water analysis, including cycles of concentration and expected chloride
-
The fouling service, and how the unit will actually be cleaned
-
Code, examination, and testing requirements
-
Physical access for maintenance and eventual replacement
Those answers settle whether the duty calls for a compact plate unit, a fabricated shell-and-tube exchanger, or a closed-loop skid — and which supplier should build it.
Harris Thermal is equipped to support custom heat-transfer and fabricated-equipment requirements on the facility side of cooling systems, where the application fits its design and fabrication capabilities. That work includes custom shell-and-tube heat exchangers built to ASME and TEMA standards, ASME Section VIII, Division 1 pressure vessels and tanks, and heat exchangers and vessels for cooling-water and water-treatment service. This is equipment that sits on the facility side of a thermal system rather than inside the packaged units it serves.
What transfers is the engineering behind it. In-house thermal and mechanical design sits alongside high-alloy fabrication in stainless, duplex, 6-moly, titanium, and nickel alloys, shop fabrication under one roof, and nondestructive examination with positive material identification. For existing plants, retubing and heat exchanger re-rating address the exchanger that has to grow with a load never in the original basis, and the TEMA classifications guide covers what each class controls in a fabricated unit.
FAQs about liquid cooling for AI data centers
What is a coolant distribution unit, and what does it actually do?
A CDU is the interface between the IT-side technology cooling system and the building’s facility water system. It holds a liquid-to-liquid heat exchanger, pumps, filtration, and controls. ASHRAE credits it with six functions, of which two matter most: separating the two loops’ water chemistry, and holding TCS supply above the room dew point to prevent condensation.
What is the difference between the technology cooling system and the facility water system?
The TCS is the IT-side loop — cold plates, manifolds, and the piping serving specific hardware. The FWS is the building or campus loop feeding chillers, towers, and coolers. The distinction matters because their water chemistry requirements differ by roughly an order of magnitude, and ownership usually splits between facilities and IT at that boundary.
Why are plate heat exchangers commonly used inside CDUs?
Because the duty is a very close approach in a very small envelope, with clean filtered fluid on both sides. Corrugated plates produce high turbulence and high surface area per unit volume. ASHRAE describes CDU exchangers as flat plate construction with 2 to 8 mm plate spacing, and commercial liquid-to-liquid units are rated at 4–5°C (7–9°F) approach.
When might a shell-and-tube heat exchanger be used in data center cooling infrastructure?
Not inside the CDU. It becomes a candidate further out where a specific condition justifies it: ASME Section VIII stamped construction, pressures or temperatures outside plate and gasket envelopes, alloys unavailable in plate form, awkward geometry, or a service life built on mechanical cleaning and retubing. Otherwise gasketed plate equipment is usually the better answer.
What water chemistry considerations apply to data center liquid cooling?
The technology cooling loop is held far tighter than the facility loop, because it wets microchannel cold plates next to energized electronics. ASHRAE puts TCS chloride under 5 ppm against under 50 ppm for facility water, with a biocide required. Open facility loops concentrate chloride by evaporation, so a makeup water analysis is not a loop water analysis.
What are the ASHRAE W-classes and S-classes?
W-classes describe facility water supply temperature — W17 through W+, with a common minimum of 2°C (35.6°F). S-classes are newer and describe technology cooling supply temperature to the IT equipment. TC 9.9 members have confirmed the S-class range was extended down to S-20, or 68°F (20°C), because high-power processors need colder coolant at the chip.
