Want Smarter and Cooler Systems? Just Add Water.


Let’s face it. Using water to cool computer chips is nothing new.

Once upon a time, such technology was applied to large systems like the IBM System/370 mainframes to cool the IBM 3081 Processor Unit. The IBM 3087 Coolant Distribution Unit incorporated heat exchangers, controls and two water pumps to supply cooling water in a closed loop to the IBM 3081 Processor Unit.

So, as we approached the Smarter Computing era, what changed in the past 30 years?

From their inception in the early 1980s, liquid-cooling technologies advanced in various ways through the 1990s. When considering water as a coolant, most commercial systems adopted a passive, cold-water-based loop structure. Expensive parts and upfront infrastructure costs were always at the other side of the equation, and water-cooling technology never really took off. It stayed restricted to a very niche market.

The outlook has become more positive in recent years. In 2009, IBM started a research project to address some of trickiest modern data center challenges, aiming to reduce power consumption and carbon emission numbers. The so-called Aquasar project consists of a water-cooled supercomputer that uses 40 percent less energy when compared to an equivalent air-cooled system. The project started as a collaboration between IBM and the Swiss Federal Institute of Technology Zurich (ETH) and had a strong appeal toward energy reuse. In addition to the energy reduction, the Aquasar system is able to take the heat generated by its compute servers and use it to warm the nearby ETH Zurich’s buildings.

Aquasar Vision: Zero-Emission Datacenter

The technology behind Aquasar leverages micro-channel liquid coolers attached directly to the processors, in specially designed IBM BladeCenter servers. The novelty of the direct chip-level cooling approach allowed higher water temperatures (up to 60 degrees Celsius) in the loop to efficiently transport heat and cool the processors to temperatures below the 85 degrees Celsius safety threshold.

The success of this first-of-a-kind project led IBM to pursue a business model to bring direct hot-water cooling technology to the market. Not very long after Aquasar, IBM started a new collaboration with the Leibniz Supercomputing Center (LRZ) to build one of the world’s fastest commercially available hot-water-cooled supercomputers, the “SuperMUC system.”

The system was built using a new direct water-cooled version of the traditional IBM System x iDataPlex dx360 M4 server. All major hotspots on the server board, including memory and application-specific integrated circuits (ASICs), are actively cooled by the water running through the manifold loops.

The traditional fans for air cooling are present only to cool the storage disks and chassis power supplies. The new compact form factor for the dx360 M4 water-cooled nodes allows for huge performance gains at the processor level. The processors can run at turbo boost speeds thanks to the hot-water cooling capability, which removes heat 4,000 times more efficiently than air. And in comparison to an air-cooled machine with the same configuration, these hot-water-cooled servers consume 40 percent less energy, resulting in substantial energy savings in a petaflop-size system like SuperMUC.

The new LRZ SuperMUC kickoff happened in June 2012, just in time to make it onto the TOP500 list of the world’s supercomputers, which was announced in June at the International Supercomputing Conference. At the time, SuperMUC entered the list in the fourth position, delivering close to three quadrillion floating point operations per second, measured using the LINPACK benchmark—an amazing achievement from an x86 based system without any GPU accelerators.

On the most current TOP500 list, announced in November 2012 at the Supercomputing 2012 (SC12) show in Salt Lake City, SuperMUC slipped a couple of positions, but still holds the sixth spot.

But what is the real purpose of the system? What are those 150,000 CPU cores going to be used for?

Among the LRZ goals is the use of the system performance to drive a wide spectrum of scientific research, from hemodynamics simulations to aircraft engineering to earth geophysical modeling. Advancements in these research areas can definitely help improve quality of life in our society. Looking back at IBM’s vision, I see Smarter Computing innovation in systems design like SuperMUC as key enablers for a Smarter Planet.

The IT infrastructure implemented in LRZ and its integration of hot-water cooling and applications-oriented, dynamic system management software means they can meet the energy requirements and actually save up to a million Euros per year, thanks to the reuse of energy to heat the buildings during winter time on the LRZ campus.

As of today, there are several manufacturers of water-cooling components and kits, and some custom computer retailers include setups of water cooling for their high-performance systems.

The show floor at SC12 was full of innovative liquid-cooling technologies, some of them already being applied to large, high-performance computing installations worldwide. In fact, all six IBM entries in the first 10 positions of the November TOP500 list leverage direct water-cooling technologies. And I’m talking about three different system architectures: Blue Gene/Q, Power 775 and iDataPlex dx360 M4.

Top 10 Supercomputers – November 2012

Top 10 Supercomputers - November 2012

Source: www.top500.org

Water cooling is back in the game.

Rodrigo Garcia da Silva is the Technical Computing Solutions Architect for IBM Systems and Technology Group in Brazil. He joined IBM in 2007 and has a total of 10 years of experience in the IT industry. You can find Rodrigo on Twitter: @rgarciatk and on LinkedIn

Redbooks Thought Leader

Smarter Computing Analyst Paper - HurwitzTo effectively compete in today’s changing world, it is essential that companies leverage innovative technology to differentiate from competitors. Learn how you can do that and more in the Smarter Computing Analyst Paper from Hurwitz and Associates.

Smarter Computing Analyst Paper - HurwitzTo effectively compete in today’s changing world, it is essential that companies leverage innovative technology to differentiate from competitors. Learn how you can do that and more in the Smarter Computing Analyst Paper from Hurwitz and Associates.

Subscribe to the Smarter Computing Blog

Recent Posts

Think parking in the city is expensive? Try maintaining a data center!

Paul DiMarzio

The costs of space and power—the two essential elements of any data center—can break a company’s desire to locate anywhere close to a big city if its business model relies on plentiful, affordable computing capacity.

Continue reading

High performance computing Q&A with the experts

Herbert Schultz

With the International Supercomputing Conference taking place this week in Leipzig, Germany, three high performance computing (HPC) industry leaders weigh in on the impact that big data is having on supercomputing, as well as other challenges and opportunities that face the market.

Continue reading

One Response to Want Smarter and Cooler Systems? Just Add Water.

  1. Water cooling system says:

    Nice image you have posted here. The way you have described here is really interesting and amazing. Water cooling system is used to cool the system from heat.

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>