Data Center Challenges
As AI servers drive unprecedented growth, data centers are now facing three key challenges: power demand, cooling efficiency, and space and operations design.
These aren’t just technical upgrades — they call for a complete rethink of how data centers are designed and managed.
Let’s break them down one by one.
1. Data Center Challenge #1: Power Density Is Soaring — Traditional Systems Are Falling Behind
AI clusters are pushing data center power systems to their limits.
A typical rack used to draw 5–10 kW, but AI racks now average 20–30 kW — sometimes far more.
That means existing PDUs, busways, and UPS units are no longer sufficient, and entire electrical backbones must be upgraded.
Many facilities that once ran on single-phase power now need three-phase systems to handle the load.
AI servers also demand redundancy — dual power feeds (A and B), each fully capable of sustaining the rack’s maximum draw.
At scale, this change is staggering.
A few years ago, a 30 MW facility was considered large; today, 200 MW AI data centers are becoming the new norm. Few regional grids can deliver that much power directly, which is why proximity to substations or generation plants has become a top factor in site selection.
2. Data Center Challenge #2: Air Cooling Can’t Keep Up — Liquid Cooling Is Taking Over
Image Source：Japan Insights
Cooling has become the second-biggest challenge for AI data centers.
Once rack power draw hits around 30 kW, air cooling reaches its practical limit.
Beyond that, even faster fans can’t remove enough heat — hotspots form easily, and hardware throttling follows.
Modern AI GPUs like NVIDIA’s H100 and B100 draw 700–1000 W each.
A rack with four 8-GPU servers can consume 50–60 kW, far beyond what traditional airflow can handle.
That’s why liquid cooling has emerged as the industry’s go-to solution. It comes in three main forms:
- Rear Door Heat Exchanger (RDHX)：Water-cooled exhaust panels, easy retrofit, handles 40–60 kW.
- Direct-to-Chip Cooling：Coolant flows through cold plates on GPUs/CPUs, supporting 60–120 kW.
- Immersion Cooling：Servers fully submerged in dielectric fluid, supports over 100–150 kW but requires complete redesign — ideal for new high-density builds.
These systems all aim for “at-source” heat removal, improving efficiency, lowering PUE, and boosting hardware stability.
Still, liquid cooling comes with trade-offs
Compared to air cooling, liquid cooling involves significantly higher upfront investment — the installation of pipelines, coolant distribution systems, and redundant components requires substantial capital expenditure.
In addition, liquid cooling systems demand regular maintenance, including coolant replacement, pipe cleaning, filter changes, and corrosion prevention, all of which introduce extra operational costs and technical complexity over the system’s lifecycle.
That’s why many facilities still stick with air cooling — especially those not yet running large-scale AI workloads.
3. Data Center Challenge #3: Space Design and Operations Are Getting Far More Complex
As high-power racks and liquid cooling become mainstream, data centers must rethink both how they’re built and maintained.
1. Space and Structure.
Installing liquid cooling pipes and CDUs means reworking layouts — aisles need to leave room for piping, raised floors must handle circulating water, and ceilings must support heavy overhead systems.
Aisle widths, raised floors, and ceiling structures must all be re-evaluated to accommodate piping, monitoring systems, and load-bearing requirements.
At the same time, rack weight is becoming a critical design constraint.
Some liquid-cooled racks, once filled with coolant, can weigh over 2 metric tons, far exceeding traditional floor load limits, requiring installation on slab floors.
The bulky nature of immersion cooling systems further limits how efficiently space can be utilized.
The cooling tanks are large and difficult to stack, which reduces the number of servers that can be installed per unit area. This must be taken into account during the early planning stages of data hall design to avoid future expansion constraints.
2. Operations Now Demand Cross-Disciplinary Skills and Training
Adding liquid cooling means adding pumps, fluid loops, and heat exchangers — all of which make operations more complex.
Engineers now need to learn how to:
- Monitor coolant temperature, flow, and pressure
- Detect and prevent leaks
- Replace cold plates and filters safely
- Drain and refill coolant without disrupting running systems
Many server vendors won’t honor warranties once water cooling is added, so companies must work with OEMs or service partners to ensure compatibility and coverage.
4. Is Our Infrastructure Really Ready?
AI servers are forcing data centers to rethink everything — power, cooling, layout, and operations.
Liquid cooling may be the answer, but it comes with steep costs and new maintenance challenges, which is why many operators are still holding back.
Before embracing high-density computing, companies need to assess total cost, scalability, and long-term resilience.
Those that plan ahead — understanding both the trade-offs and system requirements — will be the ones to build future-ready, efficient AI infrastructure.
👉 下一篇：How AI Servers Are Changing Data Center Design Logic？
Keep in touch：Contact
References：
McKinsey & Company：https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/ai-power-expanding-data-center-capacity-to-meet-growing-demand
RCR Wireless：https://www.rcrwireless.com/20250328/fundamentals/top-ai-datacenter-power
Cushman & Wakefield：https://www.cushmanwakefield.com/en/insights/global-data-center-market-comparison
Research Institute of Southwest Securities：https://www.fxbaogao.com/detail/4508054
Datacenter Dynamics：https://www.datacenterdynamics.com/en/opinions/liquid-cooling-in-your-white-space-addressing-the-concerns-for-increased-cooling-requirements/
Leave a Reply