State-of-the-art computer systems have been instrumental in making UC Santa Cruz one of the world's leading centers for computational astrophysics and planetary science. A new supercomputer recently installed on campus provides an order of magnitude improvement in the ability of researchers to address fundamental questions in cosmology and astrophysics. Its value is further enhanced by a high-capacity data storage system for archiving and sharing the results of astrophysical simulations.
The powerful new "Hyades" supercomputer will be used by UCSC researchers to simulate phenomena such as exploding stars, black holes, magnetic fields, planet formation, the evolution of galaxies, and how structure emerged in the cosmos after the big bang. The $1.5 million machine was funded by a National Science Foundation (NSF) Major Research Instrumentation grant of $910,000, augmented by campus contributions and favorable deals from vendors such as Dell and Intel.
Paired with the supercomputer is a Huawei Universal Distributed Storage (UDS) system that provides one petabyte of high-performance storage capacity. The Huawei UDS cloud storage system, on loan to the Center for Research in Storage Systems (CRSS) at the Baskin School of Engineering, is expected to become one of the largest repositories of astrophysical data outside of national facilities. Shawfeng Dong, scientist and computing cluster administrator for the Department of Astronomy and Astrophysics, oversaw the installation and integration of the Hyades supercomputer and Huawei storage system.
Piero Madau, professor of astronomy and astrophysics and principal investigator on the NSF grant, said, "Hyades is more than ten times better than our previous machine, and with the Huawei system providing storage for our simulation results, we can maximize the value of those results by making them available to the astrophysics community."
Joel Primack, professor of physics at UCSC and director of the UC High-Performance AstroComputing Center (UC-HiPACC), explained that supercomputer simulations can generate such huge amounts of complex data that it becomes difficult to analyze them on the fly. An enormous amount of storage capacity is needed for the output of these simulations so that the results can be studied and shared with other researchers.
"The Huawei system will be used to store our astrophysics results, not only from Hyades but also from simulations that we run at the big national supercomputing facilities, such as at NASA Ames or Oak Ridge National Laboratory," Primack said. "Those facilities can only store the results for a limited time, and they also restrict access to them. Now, with the Huawei storage system, we can put our results on a local server."
The Theoretical Astrophysics at Santa Cruz (TASC) computational astrophysics group includes about 20 faculty and at least 50 postdoctoral researchers and graduate students in four departments: Applied Math and Statistics, Astronomy and Astrophysics, Earth and Planetary Sciences, and Physics. In addition, computer scientists at the CRSS will be studying the performance of the new Huawei UDS system. Huawei is among the industry sponsors of CRSS, an Industry/University Cooperative Research Center supported by NSF.
"We're interested in how scientists store and use big data in a system like Hyades," said CRSS executive director Andy Hospodor. "We have studied other operating environments and are very interested in learning about astrophysical data. Our faculty and students will find ways to improve the performance, reliability, and energy efficiency of such large-scale data systems."
With its petabyte storage capacity, the Huawei system provides an enormous increase in the data storage capabilities of UCSC's computational astrophysics group. Huawei has provided a similar UDS cloud storage system for CERN, the European particle physics lab in Switzerland, to handle data from the Large Hadron Collider. UDS enables storage and sharing of big data on its mass object-based storage infrastructure, which employs high-density and energy-saving storage nodes based on ARM architecture, intelligently adjusts the workload at each node, and grows evenly with each capacity expansion.
The new Hyades supercomputer is much more powerful than the one it replaces (called Pleiades), but it occupies the same space and uses the same power in the UCSC data center. Hyades features 376 Intel Sandy Bridge Xeon CPUs (3008 x86_64 cores in total), 8 Nvidia K20 GPU computing accelerators, 3 Intel Xeon Phi 5110P accelerators, and 13 terabytes of memory. It has a peak speed of 60 teraflops (a teraflop is one trillion floating-point operations per second).
This mid-size high-performance computing system will support cutting-edge science and the training of the next generation of computational astrophysicists. It also enables researchers to hone their tools for high-resolution three-dimensional simulations run on much larger systems off campus. By demonstrating the efficiency of their codes on campus facilities, UCSC faculty successfully compete for time on some of the world's most powerful machines, such as those owned by NSF, NASA, and the Department of Energy.
"Having a local computing cluster is very important for developing the code to run on these supercomputers. They're hard to program, and having a local machine gives us a big leg up," Primack said.
